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FOREWORD 


Opinions,  interpretations,  conclusions  and  recommendations  are 
those,  of  the  author  and  are  not  necessarily  endorsed  by  the  U.S. 
Army. 


_  Where  copyrighted  material  is  quoted,  permission  has  been 

obtained  to  use  such  material. 

_  Where  material  from  documents  designated  for  limited 

distribution  is  quoted,  permission  has  been  obtained  to  use  the 
material. 

_  Citations  of  commercial  organizations  and  trade  names  in 

this  report  do  not  constitute  an  official  Department  of  Army 
endorsement  or  approval  of  the  products  or  services  of  these 
organizations . 

_  In  conducting  research  using  animals,  the  investigator (s) 

adhered  to  the  "Guide  for  the  Care  and  Use  of  Laboratory 
Animals,”  prepared  by  the  Committee  on  Care  and  Use  of  Laboratory 
Animals  of  the  Institute  of  Laboratory  Resources,  National 
Research  Council  (NIH  Publication  No.  86-23,  Revised  1985) . 

_  For  the  protection  of  human  subjects,  the  investigator (s) 

adhered  to  policies  of  applicable  Federal  Law  45  CFR  46. 

_  In  conducting  research  utilizing  recombinant  DNA  technology, 

the  investigator (s)  adhered  to' current  guidelines  promulgated  by 
the  National  Institutes  of  Health. 

_  In  the  conduct  of  research  utilizing  recombinant  DNA,  the 

investigator (s)  adhered  to  the  NIH  Guidelines  for  Research 
Involving  Recombinant  DNA  Molecules. 

_  In  the  conduct  of  research  involving  hazardous  organisms, 

the  investigator (s)  adhered  to  the  CDC-NIH  Guide  for  Biosafety  in 
Microbiological  and  Biomedical  Laboratories. 


I.  INTRODUCTION 


A.  SUBJECT  OF  STUDY 

From  FY90  to  FY97,  the  attrition  rate  Of  U.S.  Navy  recruits  increased  from  8.9%  to  13.4%.  Attrition 
data  analysis  found  psychological  disorders  to  be  the  most  significant  cause  for  female  attrition.  Males, 
on  the  other  hand,  were  separated  mostly  for  disciplinary  and  legal  offenses.  Training  experts  in  the 
Navy  know  that  academic  difficulties  are  often  manifested  in  problem  behaviors  (adjustment  disorders, 
disciplinary  and  medical)  which  lead  to  separation  from  the  Service.  To  stem  attrition,  maximize  the 
acculturation  process,  and  align  training  to  address  Fleet  requirements,  the  Navy  is  revamping  its 
training  environment,  instructional  designs,  and  pedagogical  practices.  But  also  recognizing  that  the 
Navy  operational  and  training  environments  have  been  male-dominated,  there  is  concern  that  gender 
integration  in  training  has  not  been  appropriately  accommodated  in  those  efforts  (i.e.,  learning  styles, 
stress,  self-efficacy,  etc.).  This  concern  is  the  subject  of  this  study. 

To  resolve  this  concern,  the  Navy  has  committed  to:  (1)  exploring  the  question  of  whether  females,  in 
particular,  are  experiencing  high  levels  of  stress  that  negatively  impact  their  ability  to  achieve  training 
success  on  a  par  with  their  male  counterparts;  and  (2)  investigating  instructional  approaches  and 
associated  technologies  that  would  mitigate  gender  differences. 

B.  PURPOSE  OF  STUDY 

The  purpose  of  this  research  is  to  increase  the  academic  success  of  female  recruits  in  a  technical  aspect 
of  recruit  training  where,  historically,  female  recruits  have  performed  less  satisfactorily  than  male 
recruits.  Efforts  include  the  development  of  an  instrument  that  measures  role  and  affective  stress,  and 
two  training  interventions:  the  Role  Model  treatment  and  the  Advance  Organizer  treatment.  The  Role 
Model  treatment  addresses  low  expectations  of  success  in  stressful  work  situations  such  as  firefighting, 
lack  of  experience  serving  in  the  various  roles  required  of  team  members,  and  differences  in 
communication  and  leadership  strategies.  The  Advance  Organizer  identifies  and  accommodates 
behaviors  which  reflect  cognitive  learning  strategies.  It  is  postulated  that  these  training  interventions 
will  improve  self-efficacy  and  training  performance.  Success  in  the  training  arena  would  thus  reduce 
stress  measurably. 

Accordingly,  the  study  was  designed  to  test  three  major  null  hypotheses: 

•  Providing  women  with  models  of  effective  performance  will  not  reduce  stress  and  will  not 
improve  job  performance. 

•  Providing  women  with  aids  to  structure  the  learning  process  will  not  reduce  stress  and  will  not 
improve  job  performance. 

•  Providing  men  and  women  with  models  of  effective  performance  will  not  result  in  male/female 
attitudinal  changes. 
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C.  SCOPE  OF  RESEARCH 


This  research  effort  began  in  FY97.  In  the  second  year,  FY98,  the  scope  included: 

•  Producing  the  videotape  footage. 

•  Recording  the  audio. 

•  Programming  the  embedded  criterion  measures. 

•  Programming  the  two  computer-based  treatment  interventions. 

•  Piloting  the  interventions  to  determine  their  viability. 

•  Collecting  and  analyzing  initial  data  to  ensure  the  appropriate  direction  of  the  study. 

•  Preparing  the  Year  Two  annual  report. 

D.  BACKGROUND  OF  PREVIOUS  WORK 

During  Year  One  of  the  project,  the  research  team  analyzed  the  training  needs,  requirements,  and 
problems  associated  with  firefighting  training  of  male  and  female  recruits.  The  research  team  developed 
the  instructional  treatment  plan,  finalized  the  research  design,  and  designed  the  instructional 
interventions.  Research  was  conducted  in  the  areas  of  cognitive  science,  learning  theories,  team 
competencies,  communications,  safety,  teamwork,  and  leadership  skills,  team  process  measures,  and 
gender  differences  in  personal  skills  development.  This  research  provided  a  theoretical  basis  for  the 
instructional  approach  taken  in  the  two  treatments  to  achieve  the  desired  effects  of  reducing  stress  and 
changing  attitudes  in  male  and  female  recruits. 

During  Year  Two  of  the  project,  the  instructional  interventions  were  developed,  the  Stress  Profile 
questionnaire  was  revised,  the  interventions  were  piloted,  and  the  data  from  the  pilot  test  was  analyzed. 

E.  INSTRUCTIONAL  INTERVENTIONS 

The  two  instructional  interventions  developed  are  described  below.  One  used  a  role  model  strategy  and 
the  other  used  an  advance  organizer  strategy.  The  two  interventions  are  computer-based  and  use 
interactive  multimedia  presentations.  Appendix  C  provides  full-color  screen  displays  from  the  two 
interventions. 

1.  Role  Model  Treatment 

The  Role  Model  treatment  was  developed  to  help  recruits  develop  appropriate  attitudes  regarding 
firefighting  and  the  ability  of  females  to  perform  in  teams. 

a.  Objectives  for  Role  Model  Treatment 

Our  research  indicated  that  most  recruits  had  not  actually  considered  that  firefighting  would  be  part  of 
their  Navy  experience.  Furthermore,  very  few  knew  what  behaviors  would  lead  to  success  in  situations 
requiring  teamwork,  especially  in  highly  stressful  situations.  In  teams,  such  as  the  fire  party,  they  were 
likely  to  rely  on  strategies  they  knew  and  those  strategies  may  not  be  the  most  effective.  The  Role 
Model  treatment  incorporated  human  modeling  techniques  which  were  expected  to  influence  the 
students  to  imitate  the  model's  behavior,  or  more  precisely,  imitate  the  model’s  choice  of  actions.  It 
was  expected  that  the  learner  would  acquire  an  attitude  which  reflects  the  expressed  or  demonstrated 
activities  of  the  human  model. 
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Upon  completion  of  the  Role  Model  treatment,  recruits  would: 


•  Provide  firefighting  information  to  members  of  the  team  clearly,  efficiently,  and  forcefully 
within  Navy  parameters. 

•  Use  proper  terminology  and  Navy  standard  communication  procedures  up  and  down  the  chain  of 
command. 

•  Give  orders  with  confidence,  which  is  the  product  of  knowledge  and  experience. 

•  State  why  they  must  learn  to  use  protective  breathing  devices  and  firefighting  equipment. 

•  Follow  directions  during  firefighting  exercises. 

•  Explain  why  they  must  refrain  from  inappropriate  behavior,  such  as  horseplay  or  unnecessary 
talking,  during  firefighting  exercises. 

•  State  why  it  is  important  to  learn  the  safety  hazards  of  firefighting  equipment. 

•  Identify  examples  of  monitoring  others’  performance  and  providing  help  when  it  is  needed. 

•  Take  and  provide  criticism  and  feedback  objectively  and  constructively. 

•  Show  situational  awareness  by  identifying  fire  hazards. 

•  Show  situational  awareness  by  identifying  egress  routes. 

b.  Design  of  Role  Model  Treatment 

The  content  of  the  Role  Model  treatment  was  realistic  firefighting  scenarios  in  which  examples  of 
appropriate  and  inappropriate  behaviors  were  demonstrated  in  the  four  areas  of  communication,  safety, 
teamwork,  and  leadership  skills.  Multimedia  presentations  with  voice-over  narration  were  used  to  point 
out  specific  examples  of  the  knowledge,  skills,  and  attitudes  related  to  these  four  areas  that  the  recruit 
should  develop  during  firefighting  training. 

The  presentation  introduced  male  and  female  sailors  recounting  their  experiences  on  board  ships.  Each 
sailor  introduced  and  narrated  a  short  video  sequence  that  discussed  examples  of  communication, 
safety,  teamwork,  or  leadership  skills. 

On-line  exercises  were  provided  in  which  the  recruit  was  required  to  identify  compliance  and  non- 
compliance  with  good  leadership,  communication,  teamwork,  and  safety  skills.  Sixty-one  (61)  full- 
motion  video  segments  were  developed  for  these  exercises. 

It  was  expected  that  recruits  who  received  the  Role  Model  treatment  would  develop  a  sense  of  both 
team  efficacy  and  self-efficacy,  which  would  prepare  them  for  firefighting  training.  The  Role  Model 
treatment  was  also  expected  to  help  them  develop  task-generic  teamwork  skills,  which  would  stand 
them  in  good  stead  throughout  their  careers. 

The  average  recruit  took  50  minutes  to  complete  the  Role  Model  treatment. 
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2.  Advance  Organizer  Treatment 


The  goal  of  the  Advance  Organizer  treatment  was  to  bridge  the  gap  between  what  the  recruits  already 
know  and  what  they  need  to  know  in  order  to  meaningfully  leam  to  fight  fires.  The  Advance  Organizer 
treatment  provided  a  context  of  meaning  for  new  information  to  be  learned.  It  was  designed  to  help 
orient  the  students  to  the  firefighting  subject  matter  in  such  a  way  that  the  subject  matter  was  directly 
related  to  any  pre-existing  knowledge  the  students  may  have  had.  It  was  expected  that  the  Advance 
Organizer  would  help  the  students  anticipate  the  performance  requirements  of  the  job  by  letting  them 
know  what  to  expect,  as  well  as  demonstrate  the  desired  behaviors  and  attitudes  for  acceptable  job 
performance. 

The  overall  objective  of  the  Advance  Organizer  was  to  provide  recruits  with  a  context  of  knowledge 
that  would  prepare  them  to  succeed  in  the  academic  and  hands-on  components  of  firefighting  training. 


a.  Objectives  for  Advance  Organizer  Treatment 

Upon  completion  of  the  Advance  Organizer  treatment,  the  recruits  would  recognize  that: 

•  Compartment  ID  and  closures  ID  are  necessary  for  reporting  and  responding  to  a  fire. 

•  Understanding  the  characteristics  and  uses  of  portable  and  fixed  fire  extinguisher  systems  is 
necessary  for  effective  firefighting. 

•  Closing  ship’s  closures  is  necessary  to  prevent  spreading  of  fire. 

•  All  sailors  must  know  the  meaning  of  ship’s  alarms. 

•  Knowledge  of  the  material  conditions  of  readiness  and  the  use  of  de-watering  equipment  is 
necessary  to  maintain  watertight  integrity. 

•  It  is  important  to  know  the  roles  and  responsibilities  of  fire  party  personnel. 

•  It  is  important  to  know  how  to  don  and  use  all  breathing  devices  and  protective  clothing. 

•  It  is  necessary  to  know  the  types  of  fire  extinguishing  materials  and  systems  that  are  used  for  the 
classes  of  fire  and  to  know  the  hazards  related  to  each. 

One  of  the  goals  for  the  design  of  the  Advance  Organizer  was  to  present  a  conceptual  framework  for 
the  new  knowledge  and  skills  presented  in  firefighting  training  in  ways  that  would  appeal  to  different 
learning  styles.  In  designing  the  strategies,  two  broad  categories  of  learners  were  created  from  the 
different  cognitive  strategies.  Instructional  strategies  that  were  expected  to  be  effective  for  different 
learning  styles  were  applied  to  each  of  the  two  broad  categories  of  learners  as  prescriptions  for 
strategies  that  would  present  the  material  most  effectively  for  each  type  of  learner.  This  is  not  to  say 
that  we  expected  a  learner  to  use  one  cognitive  strategy  in  all  situations.  We  strongly  suspected  that 
learners  would  switch  strategies  to  accommodate  different  subject  areas  and  different  circumstances. 
Therefore,  the  Advance  Organizer  provided  different  instructional  modes  to  accommodate  different 
cognitive  strategies,  and  recorded  which  strategies  were  being  used  at  particular  points  in  the 
instruction. 

b.  Design  of  Advance  Organizer  Treatment 

The  Advance  Organizer  used  a  combination  of  learner  control  and  intelligent  tutoring  with  assessment 
of  student  requirements  and  presentation  of  appropriate  material  to  provide  a  unique  instructional 
experience  to  each  student.  The  system  presented  information  in  ways  that  specifically  addressed  the 
style  and/or  concerns  of  the  learner. 
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(1)  Learner  Control 


In  addition  to  the  typical  learner  control  features  such  as  pacing  and  review,  the  learner  can  select  two 
different  expository  approaches:  a  structured  instructional  approach  (tutorial),  or  an  approach  in  which 
the  learner  imposes  his  or  her  own  structure  (concept  map).  Given  the  learning  preferences  indicated  in 
the  research,  it  was  expected  that  a  field-dependent/verbal/linear/operation  learner  would  prefer  the 
tutorial,  while  a  field-independent/ spatial/holistic/comprehension  learner  would  prefer  the  concept  map. 

(a)  Selection  of  Tutorial 

If  the  student  selected  the  tutorial,  this  indicated  his  or  her  preference  for  structured  instructional 
sequencing  and  a  verbal  overview  of  the  topic.  Since  the  student  had  indicated  that  he  or  she  was  a 
field-dependent  learner,  the  system  structured  the  lesson  for  the  student  from  the  specific  to  the  general. 

(b)  Selection  of  Concept  Map 

If  the  student  selected  the  concept  map,  this  indicated  his  or  her  preference  to  have  control  over  the 
sequencing  of  information,  thus  exhibiting  characteristics  of  a  field-independent  learner.  Since  the 
student  had  indicated  that  he  or  she  was  a  field-independent  learner,  the  system  provided  a  graphic 
overview  of  firefighting  in  the  form  of  a  multimedia  concept  map.  Once  the  introduction  was  over,  the 
student  was  able  to  select  specific  topics.  In  doing  so,  the  student  was  deriving  his  or  her  own  structure 
of  the  information  and  exercising  learner  control  over  the  structure  and  sequencing  of  the  information. 
The  concept  map  structure  fits  the  field-independent  learning  style  by  providing  a  “big  picture”  of  the 
lesson  material  before  the  student’s  attention  turns  to  the  details  of  the  subject. 

(2)  Learner  Questions 

The  student  can  ask  the  system  questions  which  are  provided  on  each  expository  screen  in  the  Advance 
Organizer,  regardless  of  whether  the  user  is  accessing  the  screen  via  the  linear  tutorial  or  the  course 
map.  The  purpose  of  the  questions  is  twofold.  First,  they  provide  individualized,  personalized 
instruction  within  the  context  of  the  computer-based  training  environment.  Questions  and  answers  are 
specific  to  the  content  in  the  lesson.  Second,  they  provide  a  basis  for  automatically  tailoring  the 
information  presented  in  the  lesson  to  the  user,  based  on  the  computer’s  assessment  of  the  user’s 
concerns.  There  are  four  categories  of  questions:  gender,  information,  role,  and  remember.  If  the  user 
asks  more  than  one  question  in  a  category  as  he  or  she  proceeds  through  the  presentation,  additional 
information  relevant  to  that  category  is  automatically  presented  for  subsequent  content  areas.  For 
example,  the  user  may  indicate,  by  asking  questions  in  the  gender  category,  that  she  is  concerned  about 
whether  females  can  be  effective  firefighters.  After  she  has  asked  two  “gender”  questions,  information 
on  females  in  firefighting  is  provided  in  subsequent  content  areas, 

Table  1  shows  the  four  categories  of  questions,  the  sample  questions,  the  instructional  concerns  which 
the  questions  are  designed  to  address,  and  the  purpose  of  providing  answers  and  additional  information 
in  the  Advance  Organizer  treatment. 

The  learner  questions  are  answered  using  multimedia  presentations  with  motion  video,  audio,  still 
photographs,  and  text,  and  the  presentation  is  continued.  Throughout  the  instruction  the  learner  retains 
control  of  the  pace  and  presentation.  The  learner  can  take  the  test  at  any  time. 

The  average  recruit  took  60  minutes  to  complete  the  Advance  Organizer  treatment. 
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Table  1.  Advance  Organizer  Learner  Questions 


Category 

Sample  Question 

User  Concern 

Purpose 

Gender  . 

Is  the  weight  of  the 
equipment  a  problem  for 
female  sailors? 

Recruit  is  showing  concern 
about  the  ability  of  females 
to  perform  successfully  in 
firefighting. 

Information  is  designed  to 
bolster  confidence  in  males 
and  females  that  females 
can  be  successful  at 
firefighting. 

Information 

How  is  this  like  something 

I  already  know? 

Can  you  explain  the  ID 
system  further? 

Recruit  is  asking  for 
additional  information 
specific  to  the  topic. 

In  some  cases  the  recruit  is 
asking  the  program  to  help 
relate  new  information  to 
known  information. 

Information  is  designed  to 
help  users  who  need 
additional  tutoring  to 
understand  concepts. 

Role 

Why  is  this  information 
important  to  me? 

Recruit  is  exhibiting  a 
concern  for  how  the 
activities  relate  to  the 
common  good.  This  is  a 
typical  concern  for  a  field- 
dependent  learner. 

Information  is  designed  to 
provide  users  with 
information  that  will  help 
them  understand  their  role 
in  firefighting  teams. 

Remember 

What  exactly  do  I  have  to 
remember  about  this  topic? 

Recruit  wants  to  know 
exactly  what  information  is 
important  to  remember. 

This  is  a  typical  concern 
for  a  field-independent 
learner. 

Information  is  a  synopsis  of 
the  content  area,  which  the 
user  can  use  as  a  mental 
checklist  for  what  is 
important  to  remember. 

6 


II.  EXPERIMENTAL  METHODS  AND  RESULTS 


A.  EXPERIMENTAL  METHODS 

The  procedures  for  the  study  were  drawn  from  the  systems  approach  to  training  development  and  the 
instructional  system  development  model.  Described  below  are  the  statistical  analyses  performed  to 
analyze  the  data,  the  procedures  for  development  and  evaluation  of  the  interventions,  and  the 
procedures  for  revision  of  the  Stress  Profile  questionnaire. 

1.  Statistical  Analysis  Methods 

The  following  statistical  analysis  methods  were  performed. 

1 .  Independent  sample  t-tests  were  performed  to  evaluate  female-male  differences  on  pre-training 
variables. 

2.  A  one-way  analysis  of  variance  (ANOVA)  with  treatment  (Advance  Organizer,  Role  Model, 
control)  as  the  group  classification  variable  was  performed  to  assess  the  effectiveness  of  the 
randomization  procedures.  The  dependent  variables  were  Armed  Forces  Qualification  Test 
(AFQT)  and  pre-training  stress  measures. 

3.  Paired  sample  t-tests  were  performed  to  describe  changes  in  stress  levels  during  firefighting 
training. 

4.  Analyses  of  covariance  were  conducted  to  provide  an  overall  test  of  the  effects  of  gender  and 
treatment  on  stress  and  performance.  Gender  (female-male)  and  treatment  (Advance  Organizer, 
Role  Model,  control)  were  the  grouping  variables  in  these  analyses.  These  variables  were 
treated  as  fixed  factors.  The  covariates  in  the  analysis  depended  on  the  dependent  variable  being 
analyzed:  AFQT  for  academic  performance  and  pre-training  scores  for  role  ambiguity,  role 
conflict,  self-efficacy,  positive  mood,  and  negative  mood.  A  separate  analysis  was  conducted 
for  each  criterion  variable. 

5.  Residualized  scores  were  computed  for  each  dependent  variable  to  analyze  patterns  of  change  in 
more  detail.  The  residualized  scores  were  computed  by  regressing  academic  test  scores  on 
AFQT  and  each  stress  indicator  on  the  corresponding  pre-training  score.  The  residualized  score 
was  the  difference  between  the  actual  post-training  score  and  the  predicted  post-training  score. 
Treatment  effects  then  were  computed  by  determining  the  average  residualized  scores  for 
specific  groups  (e.g.,  females  receiving  the  Advance  Organizer  treatment).  The  analyses  of 
residualized  scores  included: 

•  Analyses  of  variance  to  provide  focused  tests  of  the  effects  of  the  Advance  Organizer  (AO) 
and  Role  Model  (RM)  treatments.  In  each  case,  a  2  x  2  ANOVA  was  performed  with 
gender  as  one  group  classification  variable.  The  other  group  classification  variable  was  an 
AO  contrast  or  an  RM  contrast.  The  AO  contrast  was  constructed  by  combining  the  RM 
and  control  groups  into  a  single  group  with  the  AO  group  as  the  second  group.  The  RM 
contrast  was  defined  the  same  way,  except  with  RM  in  place  of  AO.  Where  appropriate, 
these  ANOVAs  were  followed  by  t-tests  for  independent  groups  to  clearly  isolate  which 
groups  differed  from  which  others.  These  tests  are  described  in  Section  ILB,  Results  of 
Data  Analysis. 
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•  Pearson  product-moment  correlations  between  the  residualized  performance  score  and  the 
residualized  stress  scores.  The  residualized  performance  score  was  an  index  of  whether  the 
person  performed  better  than  expected  based  on  his  or  her  ability  (positive  residual)  or 
worse  than  expected  (negative  residual).  The  residual  stress  scores  indicated  whether  post¬ 
training  stress  was  higher  than  expected  based  on  anticipatory  scores  (positive  stress 
residual)  or  lower  than  expected  (negative  stress  residual).  Exceptionally  high  stress  would 
be  associated  with  poorer  performance  if  the  correlations  were  negative  for  role  ambiguity, 
role  conflict,  and  negative  mood.  In  these  cases,  a  positive  residual  implies  high  stress,  so  a 
negative  correlation  links  high  stress  with  lower  than  expected  performance.  In  the  cases  of 
self-efficacy  and  positive  mood,  negative  correlations  would  indicate  that  poor  performance 
was  associated  with  stress.  This  reversal  occurs  because  lower  self-efficacy  and  positive 
mood  indicate  stress. 

2.  Development  Methods  for  Interventions 

During  Year  Two  of  the  project,  the  interventions  were  developed  and  the  Stress  Profile  questionnaire 
was  revised.  The  methods  for  development  of  the  treatments  consisted  of  the  following  tasks. 

1 .  The  AO  and  RM  storyboards  (design  documents)  were  reviewed  and  approved  by  the  Navy. 

2.  The  lists  of  required  media  elements  (graphics,  video,  photographs,  audio)  to  effectively  present 
the  content  were  generated  for  the  two  interventions. 

3.  Existing  videotapes  were  obtained  from  the  Navy  and  added  to  those  obtained  at  the  Recruit 
Training  Center  (RTC)  at  Great  Lakes  Naval  Base  to  incorporate  realistic  footage  where 
possible  in  the  AO  and  RM  interventions.  The  incorporation  of  existing  video  added  to  the 
realism  of  instructional  scenes.  Existing  video  footage  (approximately  14  videotapes)  was 
reviewed  for  content.  Specific  frames  were  identified  for  use  primarily  in  the  Advance 
Organizer. 

4.  Basic  database  structures  and  variables  were  designed  to  gather  user  interaction  data  for  both 
instructional  interventions.  Authorware  was  chosen  as  the  programming  language  for  the 
computer-based  interventions  and  the  data  was  saved  as  delimited  text  files  that  could  be  viewed 
in  several  types  of  software.  (After  the  evaluation,  Microsoft  Access  was  used  to  organize  the 
data  for  statistical  analysis.) 

5.  During  a  two-week  time  period,  Southwest  Research  Institute  (SwRI)  and  the  U.S.  Navy  video 
production  crew  videotaped  content  for  the  Role  Model  treatment  at  Great  Lakes  Naval  Base. 
Two  subject  matter  experts  (SME)  from  the  RTC  were  present  for  scene  development  and 
review.  A  damage  control  (DC)  instructor  (male),  a  DC-rated  sailor  (female),  and 
approximately  10  recruits  served  as  the  talent.  In  addition  to  the  former  two  sailors,  six  other 
sailors  (petty  officers)  were  also  videotaped  for  Role  Model  testimonial  content.  SMEs 
performed  final  review  of  AO  and  RM  instructional  storyboards  for  content  and  accuracy. 

6.  The  Navy  video  crews  edited  video  segments  and  sent  footage  to  SwRI  for  further  identification 
and  digitization. 

7.  Video  footage  and  still  photographs  were  digitized  for  use  in  the  treatments. 
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8.  Professional  audio  narration  was  recorded  and  digitized  for  both  treatments.  Three  audio 
recording  professionals,  two  females  and  one  male,  were  chosen  to  record  the  narration  for  the 
Advance  Organizer  treatment,  and  one  female  narrator  recorded  the  audio  for  the  Role  Model 
treatment. 

9.  Instructional  graphics  for  inclusion  in  both  interventions  were  produced. 

10.  The  graphical  user  interface  (GUI)  was  developed  by  the  graphic  artist  and  treatment 
development  teams. 

11.  Treatments  were  programmed  in  Authorware. 

12.  Database  variables  were  incorporated  into  the  programs  and  tested  for  accuracy  in  gathering 
data. 

13.  SwRI  revised  the  consent  form  and  received  approval  from  the  Institutional  Review  Board 
(IRB).  Revisions  to  the  consent  forms  included  making  the  form  easier  to  read  and  designing  it 
to  fit  on  one  page. 

14.  SwRI  developed  the  procedures  and  checklists  to  support  data  collection  for  use  during  the 
evaluation. 

15.  The  SwRI  team  traveled  to  the  RTC  at  Great  Lakes  Naval  Base  for  implementation  of 
treatments. 

16.  The  Navy  computer  lab  was  prepared  for  data  collection.  Computers  were  tested  and  “fixed” 
(sound  cards,  headphones,  video  replay,  etc.)  by  Navy  personnel.  Treatments  were  installed  on 
46  available  computers  in  the  Fundamental  Applied  Skills  Training  (FAST)  Learning  Resource 
Center  (LRC).  AO  was  installed  on  23  computers  and  RM  was  installed  on  23  computers. 
Treatments  were  identified  on  computers  labeled  “AO”  or  “RM”. 

17.  Implementation  procedure  checklists  were  followed  during  implementation  to  ensure 
consistency  during  the  data  collection  effort. 

18.  The  treatments  were  revised  in  response  to  data  generated  during  the  pilot  evaluation.  See 
Appendix  B  for  a  list  of  the  changes  that  were  made  to  the  treatments. 

3.  Revision  of  Stress  Profile  Questionnaire 

Revision  of  the  Stress  Profile  questionnaire  was  accomplished  during  Year  Two  of  the  study,  prior  to 
implementation  of  the  interventions.  A  decision  was  made  to  repeat  some  elements  of  the  prior 
exploratory  analyses  for  the  measurement  structure  of  stress.  This  decision  was  made  because  some 
items  used  to  measure  the  stresses  of  general  recruit  training  did  not  adapt  well  to  the  specifics  of 
firefighting  training.  In  addition,  some  items  were  modified  significantly  (e.g.,  one  ambiguity  item  was 
split  into  two  distinct  items)  and  three  new  items  were  added.  These  changes  were  sufficient  to  make  it 
reasonable  to  replace  the  planned  confirmatory  analyses  with  more  exploratory  analyses. 
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4.  Procedures  for  Implementation  of  Interventions  and  Data  Collection 

The  procedures  for  implementation  of  the  interventions  and  data  collection  consisted  of  the  following 
tasks. 

1.  The  data  collection  effort  began  with  administration  of  the  consent  forms  and  Stress  Profile 
questionnaires  to  the  recruit  participants.  The  recruits  completed  the  paperwork  within  one 
hour,  in  a  classroom  located  in  the  RTC  Headquarters  building  (178)  by  division.  Two 
integrated  recruit  divisions,  233  and  234,  were  in  the  classroom  at  the  same  time.  Division  233 
consisted  of  males  and  Division  234  consisted  of  females. 

Recruit  participants  were  introduced  to  the  administrators  and  topic  of  the  study.  The  consent 
forms  were  distributed  and  read  to  the  recruits.  Questions  were  answered,  and  the 
administrators  instructed  any  recruits  who  did  not  wish  to  participate  in  the  voluntary  study,  or 
who  had  extenuating  circumstances,  to  move  to  another  area  of  the  classroom.  (Extenuating 
circumstances  included  evening  watch  duty  conflicts  and  other  schedule  conflicts.)  The  consent 
forms  were  signed  by  the  participating  recruits  and  collected.  Of  a  total  of  349  recruits  in  the 
divisions,  271  participated  in  the  study.  Seventy-eight  (78)  recruits  did  not  participate  due  to 
extenuating  circumstances. 

2.  After  completion  of  the  consent  forms,  the  Stress  Profile  questionnaires  were  distributed  to  all 
the  recruits  who  would  participate  in  the  study.  Directions  were  provided  on  how  to  fill  out  the 
scantron  forms  to  answer  the  Stress  Profile  questions.  Recruits  were  shown  how  to  fill  in  their 
Social  Security  number,  gender,  division,  name,  and  questionnaire  number  (1,  2,  or  3).  The 
first  student  in  the  first  row  was  given  number  one,  the  second  student  was  given  number  two, 
the  third  student  was  given  number  three,  the  fourth  student  was  given  number  one,  and  so 
forth.  All  of  the  students  who  were  given  the  number  one  received  the  AO  treatment.  The 
students  assigned  number  two  received  the  RM  treatment.  The  students  assigned  number  three 
became  the  control  group.  The  Stress  Profile  questionnaire  was  explained  and  students  were 
instructed  to  answer  the  questions  with  regard  to  how  they  felt  about  recruit  firefighting  at  that 
moment  in  time. 

3.  In  order  to  randomly  assign  recruits  to  treatment  or  control  groups,  after  the  recruits  completed 
the  Stress  Profile  questionnaire,  the  scantron  forms  were  divided  into  three  groups  in  relation  to 
the  questionnaire  number  indicated  (1,  2,  or  3).  The  scantron  forms  were  counted  to  ensure 
equal  cell  (sample)  size  among  treatment  and  control  groups.  Group  1  was  assigned  to  the 
Advance  Organizer  (AO)  group  and  Group  2  was  assigned  to  the  Role  Model  (RM)  group.  In 
order  to  accommodate  the  number  of  computers  available  in  the  lab  and  the  number  of  students 
receiving  treatments,  these  two  groups  were  instructed  to  return  that  evening  at  a  designated 
time  (5:40  pm  or  7:40  pm)  depending  on  their  division.  For  example,  Groups  1  and  2  from  one 
division  returned  at  5:40  pm  while  Groups  1  and  2  from  the  other  division  returned  at  7:40  pm. 
Recruits  in  Group  3  were  instructed  that  they  would  not  be  taking  the  computer-based 
exercises.  They  were  ensured  that  they  would  receive  all  of  the  same  training  that  all  other 
recruits  received  at  the  RTC.  Group  3  recruits  are  the  control  groups  for  the  data  analysis. 

4.  As  the  recruits  arrived  at  the  FAST  lab,  they  were  instructed  to  sit  at  computers  designated  by  a 
sign  that  read  “AO”  or  “RM”  depending  on  the  treatment  group  they  were  assigned  to.  For 
example,  recruits  in  Group  1  were  instructed  to  sit  at  the  computers  labeled  “AO”  and  recruits 
in  Group  2  were  instructed  to  sit  at  the  “RM”  computers.  The  administrators  told  the  recruits 
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that  there  were  two  different  computer-based  applications  in  the  lab  and  that  they  would  interact 
with  only  one  of  them.  The  administrator  explained  that  there  might  be  a  difference  in  the 
amount  of  time  it  would  take  them  to  complete  the  programs,  based  on  the  application  and  on 
the  individuals.  The  administrators  explained  that  the  recruits  should  take  their  time  while 
interacting  with  the  program,  as  they  had  ample  time  (90  minutes)  within  which  to  finish.  As 
recruits  completed  the  program,  they  were  directed  to  a  classroom  to  wait  until  the  entire 
division  was  finished. 

Recruits  were  encouraged  to  ask  questions  and  make  comments  to  the  administrators  at  any 
time  during  the  program.  The  recruits  were  told  that  we  were  evaluating  the  system  and  any 
feedback  from  the  recruits  would  be  helpful  toward  improving  the  system  for  future  uses.  See 
Appendix  A  for  specific  comments  generated  by  the  students  during  the  data  collection  effort. 

5.  The  recruits  completed  the  treatments.  The  computer  screens  started  at  the  Windows  95  desktop 
interface  with  the  “sign  on”  application  highlighted.  The  recruits  were  shown  how  to  click  with 
a  mouse  and  enter  data  using  the  keyboard.  The  administrators  instructed  the  students  as  a 
group  through  the  sign-on  application.  After  the  recruits  were  “signed  on”,  they  were 
instructed  to  begin  the  treatments.  The  administrators  emphasized  how  important  it  was  that  the 
students  not  quit  the  program,  because  all  data  would  be  lost  and  would  not  be  recoverable. 

6.  During  the  students’  interaction  with  the  program,  they  were  reminded  how  much  time  they  had 
left  beginning  at  the  60-minute  point.  The  time  left  was  announced  at  30,  20,  10,  and  5 
minutes.  When  all  recruits  had  completed  the  programs  and  were  seated  in  the  adjacent 
classroom,  a  cross-check  was  performed  on  the  names  in  each  group  and  the  actual  treatments 
the  students  had  completed.  Each  division  had  one  or  more  persons  who  had  sat  at  the  “wrong” 
computer  and  were  given  the  “wrong”  treatment.  This  data  was  collected,  which  explains  the 
differences  in  cell  sizes  between  treatments  and  control  groups. 

7.  After  dismissal  of  the  division  in  its  entirety,  the  administrators  prepared  the  lab  for  the  next 
division’s  arrival.  Administrators  returned  all  computer  screens  to  the  Windows  95  desktop 
interface  and  inserted  fresh  program  diskettes  into  the  floppy  drives  for  collection  of  the  data 
for  the  next  group  of  students. 

8.  The  same  procedures  were  followed  for  the  next  group  of  recruits.  After  dismissal  of  the 
division,  the  paperwork  was  filed,  floppy  disks  were  collected,  and  the  computers  were  reset  to 
the  Windows  95  desktop  interface  for  daytime  use  in  the  lab. 

B.  RESULTS  OF  DATA  ANALYSIS 
1.  Stress  Measurements 

Exploratory  principal  component  analyses  were  conducted  for  the  40  stress  items.  These  analyses  were 
adopted  in  preference  to  the  confirmatory  factor  analyses  planned  at  the  time  the  project  was  initiated. 
The  exploratory  analyses  were  adopted  as  a  first  step  because  a  number  of  items  had  been  included  in 
the  study  that  were  not  originally  designed  to  measure  either  role  ambiguity  or  role  conflict.  Some  new 
items  were  added  to  reflect  comments  from  recruits  about  firefighting  training.  The  additional  items 
could  affect  the  overall  structure  of  measurement  in  ways  that  would  not  be  evident  if  a  selected  set  of 
models  was  imposed  on  the  data.  On  the  other  hand,  if  the  original  measures  of  role  ambiguity  and  role 
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conflict  retained  their  structure  even  with  these  changes,  the  exploratory  approach  would  still  identify 
them.  The  exploratory  approach,  therefore,  provided  an  opportunity  to  identify  points  at  which  the 
intended  measurement  model  broke  down,  if  it  did,  but  would  still  recover  that  measurement  model  if  it 
was  appropriate. 

The  initial  model  provided  a  reasonable  basis  for  measurement.  The  structure  of  the  item  set  indicated 
the  presence  of  more  than  two  stress  dimensions,  but  the  close  approximations  to  the  original  scales 
were  recovered.  First,  the  analyses  indicated  the  presence  of  five  eigenvalues  >1.00  for  both  the  pre- 
and  post-training  data.  However,  in  each  case,  only  three  eigenvalues  were  greater  than  expected  by 
chance  according  to  the  Monte  Carlo  results  of  Cota,  Longman,  Holden,  Fekken,  and  Xinaris  (1993). 
Therefore,  three  components  were  extracted. 

Two  of  the  three  extracted  components  were  close  approximations  to  the  a  priori  role  ambiguity  and 
role  conflict  scales.  This  conclusion  was  reached  by  rotating  the  extracted  components  to  an  orthogonal 
varimax  solution.  The  rotated  components  were  matched  across  solutions  by  determining  which  three 
items  had  the  highest  loading  on  each  component.  After  matching  components,  an  item  was  designated 
as  an  indicator  for  a  component  if:  (a)  the  component  loading  was  >  .30  in  both  the  pre-training  and 
post-training  data,  and  (b)  loading  on  other  components  was  <  .30. 

Role  ambiguity  and  role  conflict  scales  were  constructed  by  combining  the  original  item  classification 
with  the  above  criteria.  Items  5,  12,  13,  16,  23,  24,  25,  and  28  met  these  criteria  for  role  ambiguity. 
Seven  of  these  eight  items  were  designed  to  measure  role  ambiguity.  Item  5  initially  was  designed  to 
measure  overload  (Vickers  &  Ryman,  1980)  and  was  included  in  this  study  as  a  potential  indicator  of 
role  conflict.  That  interpretation  of  the  item  responses  assumed  that  conflict  was  generated  by  having  to 
choose  between  one  of  several  tasks  that  needed  to  be  done  at  the  same  time.  In  the  context  of 
firefighting,  however,  this  item  appears  to  measure  the  ability  to  get  work  done  quickly  because  roles 
are  clearly  specified. 

Items  2,  9,  15,  20,  and  22  met  the  scale  construction  criteria  for  role  conflict.  These  items  comprised 
five  of  six  items  included  in  the  analyses  that  originally  were  designed  to  measure  role  conflict. 

The  third  component  in  the  analysis  was  defined  by  items  originally  designed  to  measure  several 
distinct  constructs.  Items  that  met  the  scaling  criteria  for  this  component  originally  were  designed  to 
measure  role  ambiguity,  role  conflict,  teamwork,  and  standardization  (Vickers  &  Ryman,  1980).  The 
set  of  items  lacked  any  clear  unifying  theme,  and  component  loadings  tended  to  be  inconsistent  from  the 
pre-treatment  to  post-treatment  period. 

No  scale  was  constructed  from  the  items  defining  the  third  principal  component  in  the  analyses.  The 
items  with  substantial  loadings  were  primarily  those  included  in  this  study  in  an  attempt  to  expand  the 
boundaries  of  the  role  ambiguity  and  role  conflict  concepts  in  the  context  of  firefighting  training  if 
appropriate.  Instead,  the  items  appear  to  have  been  useful  in  producing  narrower,  more  highly  focused 
scales  for  role  ambiguity  and  role  conflict.  A  scale  constructed  of  the  items  defining  this  component 
would  have  had  no  meaning  in  the  context  of  the  specific  theoretical  focus  of  the  study.  The 
inconsistent  component  loadings  raise  concerns  about  the  appropriateness  of  constructing  a  scale  from 
those  items.  Even  if  a  scale  had  been  constructed,  it  would  have  had  no  obvious  relationship  to  the 
study  hypotheses.  Those  hypotheses  dealt  with  role  conflict  and  role  ambiguity.  Acceptable  scales  for 
conflict  and  ambiguity  had  been  identified,  so  the  focal  stresses  for  the  research  were  covered. 
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Table  2.  Principal  Components  of  Stress  Items 


Item 

Component: 

2 

3 

No. 

Time: 

Pre 

Post 

Pre 

Post 

Pre 

Post 

Role  Ambiguity 

13 

Procedures  detailed 

.75 

.68 

.18 

.30 

28 

Orders  clear 

.69 

.67 

.07 

.10 

.04 

-.15 

12 

Communications  clear 

.67 

.27 

24 

Only  one  way  to  do  each  task 

.48 

.63 

-.02 

-.04 

-.07 

.08 

23 

Team  responsibilities  clear 

.75 

.62 

.07 

.21 

.04 

-.26 

25 

Goals  clear 

.62 

.57 

.21 

.31 

-.07 

-.07 

16 

Explanations  clear 

.57 

.61 

.38 

.36 

.11 

-.04 

5 

Recruits  work  fast 

.39 

.34 

-.17 

.33 

.28 

.02 

Role  Conflict 

15 

Rules  often  conflict 

.07 

-.15 

.10 

-.01 

.65 

.73 

9 

Tasks  done  by  numbers 

-.11 

-.21 

.22 

.05 

.74 

.69 

20 

Confusion  over  who  does  which  tasks 

.29 

.01 

-.22 

-.18 

.54 

.64 

22 

Difficult  to  keep  up 

.17 

.07 

-.37 

-.26 

.46 

.61 

2 

Done  differently 

.07 

.00 

.27 

.13 

.53 

.34 

Third  Component 

7 

Know  expected  standards 

.21 

.41 

.72 

.42 

.11 

.00 

3 

Recruits  cooperate 

.51 

.19 

.27 

.71 

.04 

-.03 

4 

Recruits  follow  standard  procedures 

.40 

-.05 

.40 

.70 

-.14 

-.02 

1 

Know  which  procedure 

.11 

.21 

.75 

.59 

.18 

-.07 

10 

Remember  technical  details 

.33 

.34 

.52 

.39 

.05 

-.06 

Not  Included 

8 

Conflicting  orders  common 

.32 

.26 

.24 

.48 

.28 

.00 

17 

Leader’s  decisions  good 

.69 

.23 

.02 

.58 

.12 

-.09 

Note:  See  text  for  description  of  principal  component  analysis  and  component  matching  procedures. 
“Pre”  and  “Post”  column  headings  refer  to  pre-training  and  post-training  data.  See  Appendix  D  for  the 
full  content  of  the  items. 


The  principal  component  analysis  produced  scales  suitable  for  assessing  role  ambiguity  and  role  conflict 
as  these  constructs  occur  in  the  context  of  firefighting  training.  The  role  ambiguity  construct  is  measured 
by  items  that  indicate  that  responsibilities  are  clearly  spelled  out.  Descriptive  statistics  for  the  pre-  and 
post-training  scales  are  given  in  Table  3. 
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Table  3.  Descriptive  Statistics  for  Stress,  Efficacy,  and  Distress  Scales 


Scale 

Pre-Training 

Post-Training 

r« 

t-test 

Sig. 

Mean 

SD 

Alpha 

Mean 

SD 

Alpha 

Role  Ambiguity 

2.94 

0.64 

OO 

o 

2.20 

0.57 

.79 

0.36 

18.01 

.000 

Role  Conflict 

3.00 

0.61 

.56 

2.68 

0.71 

.57 

0.21 

6.43 

.000 

Self-Efficacy 

3.78 

0.56 

.58 

3.92 

0.53 

.59 

0.29 

3.53 

.000 

Negative  Mood 

1.99 

0.77 

.80 

1.74 

0.75 

.81 

0.38 

4.82 

.000 

Positive  Mood 

3.25. 

0.81 

.82 

3.40 

0.89 

.80 

0.52 

3.01 

.003 

Note:  Table  entries  were  computed  for  participants  with  stress  and  mood  data  for  the  pre-  and  post¬ 
training  sessions.  The  sample  size  was  n  =  277  except  for  negative  mood  (n  =  265)  and  positive  mood 
(n  =  267).  Alpha  is  Cronbach’s  internal  consistency  estimate  of  reliability,  “n”  is  the  correlation  of  the 
pre-training  scores  with  the  post-training  scores  (Nunnally  &  Bernstein,  1994,  pp.  233-236). 


Firefighting  training  decreased  stress.  Role  clarity  increased  during  training  from  a  pre-training  average 
rating  very  close  to  the  “Neither  Agree  Nor  Disagree”  anchor  of  the  rating  scale  (i.e.,  3)  to  near  the 
“Agree”  anchor  (i.e.,  4).  Role  conflict  dropped  from  an  anticipatory  mean  value  of  “Neither  Agree 
Nor  Disagree”  to  a  post-training  rating  about  one-third  of  the  way  from  the  original  rating  and 
“Disagree”  (i.e.,  2). 

a.  Self-Efficacy 

Self-efficacy  measures  the  person’s  perception  that  he  or  she  can  master  the  tasks  required  in  firefighting 
training.  Self-efficacy  scores  also  include  the  perception  that  the  tasks  can  be  performed  with  relative  ease 
and/or  with  simple  persistence. 

Eight  items  in  the  Stress  Profile  questionnaire  were  designed  to  measure  self-efficacy.  Principal 
component  analysis  indicated  the  presence  of  a  small  second  factor  in  these  items,  but  the  total  set  of 
items  was  used  to  measure  the  construct.  This  procedure  corresponded  to  the  concept  of  generalized  self- 
efficacy  that  was  the  basis  for  the  development  of  the  self-efficacy  items.  The  resulting  scale  had  a  rather 
low,  but  marginally  acceptable,  internal  consistency  estimate  of  reliability  (cf.  Table  3). 

Average  self-efficacy  increased,  but  the  gain  was  modest.  The  initial  rating  was  approximately  three- 
fourths  of  the  distance  from  “Neither  Agree  Nor  Disagree”  to  “Agree.”  The  final  rating  was 
approximately  nine-tenths  of  the  distance.  The  absolute  average  gain  (0.14  rating  points)  translated  to 
an  effect  size  of  0.25  using  the  pre-training  standard  deviation  (SD)  as  the  frame  of  reference.  This 
change  represents  a  small  effect  (Cohen,  1988)  in  the  predicted  direction. 

All  of  the  differences  between  anticipatory  stress  and  post-training  stress  were  comparable  for  females 
and  males.  None  of  the  gender  by  time  interactions  was  statistically  significant  (p  >  .117  for  each). 

b.  Positive  and  Negative  Moods 

Mood  assessment  consisted  of  12  items,  six  selected  to  measure  negative  mood  and  six  selected  to 
measure  positive  mood.  Extensive  prior  research  indicates  that  these  two  general  dimensions  provide  a 
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useful  summary  of  mood  (Watson  &  Tellegen,  1985).  Vickers  and  Hervig  (1987)  showed  that  this 
model,  applied  to  military  personnel,  was  not  merely  a  response  style  artifact;  it  extracted  the  valid 
variance  from  mood  repons  in  this  population. 

Given  the  selection  of  items,  it  was  anticipated  that  the  mood  items  would  produce  two  clear 
dimensions.  Principal  components  analysis  produced  only  two  eigenvalues  >  1.00  in  both  the  pre¬ 
treatment  and  post-treatment  data  (Pre-treatment  eigenvalues  =  4.13,  2.29,  0.90;  Post-treatment 
eigenvalues  =  3.94,  2.72,  0.88). 

The  component  loadings  confirmed  the  expected  mood  structure  (Table  4).  The  six  negative  mood  items 
defined  a  single  dimension  in  each  analysis;  The  six  positive  mood  items  defined  the  other  dimension. 
Based  on  this  evidence  and  the  prior  studies  of  mood  structure,  the  items  were  combined  to  form  the  a 
priori  scales  planned  for  positive  mood  and  negative  mood.  The  resulting  six  item  scales  had  acceptable 
reliability  (cf.  Table  3). 


Table  4.  Principal  Components  Structure  of  Affective  Distress 


Item 

Component  1 

Component  2 

Pre 

Post 

Pre 

Post 

37 

Sad 

.78 

.83 

-.07 

-.07 

33 

Blue 

.74 

.70 

-.18 

-.11 

35 

Angry 

.73 

.78 

-.06 

.00 

38 

Annoyed 

.68 

.82 

-.09 

.02 

32 

Uneasy 

.68 

.58 

-.02 

.  ... 

.02 

29 

Depressed 

.66 

.68 

-.23 

-.26 

39 

Happy 

-.17 

-.17 

.79 

.80 

40 

Energetic 

-.16 

-.14 

.79 

.79 

31 

Cheerful 

-.12 

-.07 

.79 

.77 

36 

Active 

-.16 

.04 

.70 

.74 

34 

Satisfied 

-.24 

-.19 

.60 

.68 

30 

Vigorous 

.27 

.13 

.56 

.57 

Note:  The  number  in  front  of  each  adjective  indicates  the  item  number  within  the  Stress  Profile 
questionnaire. 


Moods  were  positive  overall  prior  to  firefighting  training,  but  improved  further  during  firefighting 
training.  The  assertion  that  moods  initially  were  positive  overall  is  based  on  the  observation  that  the 
average  rating  for  positive  mood  was  slightly  above  the  “Neither  Agree  Nor  Disagree”  rating  (3.25), 
while  the  average  rating  for  negative  mood  was  close  to  the  “Disagree”  anchor  point  (1.99). 

The  inference  that  mood  improved  during  firefighting  training  is  based  primarily  on  a  drop  in  the  level 
of  negative  mood.  The  average  rating  for  negative  mood  dropped  one  quarter  of  a  point  to  a  value  one- 
fourth  of  the  way  between  “Disagree”  and  “Strongly  Disagree.”  Using  the  pre-training  mood  as  the 
reference,  this  change  amounted  to  a  small  effect  size  (ES  =  0.32). 
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The  inference  that  mood  improved  was  supported  by  a  marginal  trend  toward  increased  positive  mood. 
The  average  positive  mood  rating  increased  by  approximately  one-seventh  of  a  rating  point  (0.15). 
Whether  this  increase  in  positive  mood  should  be  regarded  as  a  useful  indicator  of  change  during 
firefighting  training  is  uncertain.  The  change  was  statistically  significant  (p  <  .003),  but  the  effect  size 
was  smaller  than  that  for  negative  mood  (ES  =  0.19  vs.  ES  =  0.32).  The  positive  mood  effect  size  fell 
just  below  Cohen’s  (1988)  recommended  minimum  for  an  important  effect  (i.e.,  ES  =  0.20). 

2.  Evaluation  of  Pre-Training  Status 

a.  Randomization  Effects 

Study  participants  were  randomly  assigned  to  treatment  groups.  While  the  procedures  followed  should 
have  produced  comparable  groups,  differences  might  have  arisen  by  chance.  For  this  reason,  analyses 
would  benefit  from  the  introduction  of  appropriate  controls  for  those  variables  that  were  unevenly 
distributed  across  the  treatment  conditions. 

A  one-way  analysis  of  variance  was  conducted  to  verify  that  the  experimental  groups  were  comparable. 
The  analysis  compared  the  three  treatments.  Measures  that  described  the  individuals  prior  to  treatment 
were  the  dependent  variables. 

The  treatment  groups  did  not  differ  significantly  on  any  pre-treatment  variable.  This  statement  was  true 
for  Armed  Forces  Qualification  Test  (AFQT)  (F2.274  =  0.17,  p  <  .849)  anticipated  role  clarity  (F2.289  = 
0.98,  p  <  .377),  anticipated  role  conflict  (F2.289  =  1.54,  p  <  .216),  and  anticipated  self-efficacy  (F2.289 
=  0.86,  p  <  All).  AFQT  is  a  general  indicator  of  intelligence.  Similarly,  the  treatment  groups 
reported  comparable  initial  levels  of  affective  distress,  including  both  negative  affect  (F2.280  =  1.40, 
p  <  .249)  and  positive  affect  (F2,28o  =  0.54,  p  <  .583)^  Clearly,  the  treatment  groups  were  well 
matched  with  respect  to  mental  ability,  anticipated  stress,  and  initial  affective  stress  status. 

b.  Gender  Differences  in  Ability 

Females  (mean  =  61.54,  SD  =  19.54)  scored  higher  on  the  AFQT  than  males  (mean  =  55.94,  SD  = 
20.76).  This  difference  was  only  marginally  significant  statistically  ( t  —  1.93  ,  274  df,  p  <  .056)  using 
a  two-tailed  significance  test.  A  two-tailed  test  was  appropriate  because  no  a  priori  prediction  had  been 
made  regarding  the  direction  of  gender  differences  in  mental  ability. 

c.  Gender  Differences  in  Anticipated  Stress 

The  stress  measurements  taken  prior  to  firefighting  training  are  best  regarded  as  measures  of  anticipated 
stress.  These  measures  asked  recruits  to  indicate  what  they  believed  the  firefighting  training  would  be 
like.  This  perception  presumably  was  based  on  the  recruits’  prior  experiences  in  basic  training,  prior 
beliefs  about  the  nature  of  the  course,  and  so  on.  These  initial  stress  perceptions  obviously  were  not 
reports  of  any  actual  experience  in  firefighting  training.  For  these  reasons,  it  was  desirable  that  the 
initial  ratings  be  differentiated  from  the  actual  stress  experienced  during  the  training.  The  label 
“anticipatory”  was  applied  to  provide  this  differentiation. 

Females  and  males  had  comparable  anticipations  about  stress  in  firefighting  training  (Table  5). 
Differences  between  the  two  groups  were  not  statistically  significant  for  role  ambiguity,  role  conflict, 
self-efficacy,  and  negative  mood.  Females  reported  slightly  higher  levels  of  positive  mood,  but  this 
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trend  was  only  marginally  significant  using  a  two-tailed  significance  test.  A  two-tailed  significance  test 
for  the  difference  in  positive  mood  scores  was  appropriate  given  the  observed  direction  of  the 
difference.  The  initial  expectation  was  that  females  would  have  higher  stress  levels  (i.e.,  less  positive 
mood  in  the  case  of  this  indicator). 


Table  5.  Pre-Training  Gender  Differences 


Scale 

Females 

Mean 

SD 

Males 

Mean 

SD 

t 

Sig. 

Role  Ambiguity 

2.72 

0.68 

2.65 

0.65 

0.79 

.433 

Role  Conflict 

2.95 

0.60 

0.83 

.405 

Self-Efficacy 

3.79 

0.49 

3.77 

0.57 

-0.33 

.744 

Negative  Mood 

1.94 

0.68 

1.76 

0.77 

0.88 

.381 

Positive  Mood 

3.42 

0.77  • 

3.20 

0.82 

-1.95 

.052 

3.  Evaluation  of  Post-Training  Status 
a.  Academic  Performance 

The  average  female  (mean  =  3.94,  SD  =  0.44)  scored  lower  on  the  examination  than  the  average  male 
(mean  =  4.15,  SD  =  0.45).  The  difference  was  statistically  significant  (r  =  3.56,  296  df,  p  <  .001). 
The  lower  average  score  for  females  was  obtained  even  though  they  had  higher  than  average  AFQT 
scores.  Because  higher  AFQT  scores  generally  were  associated  with  higher  examination  scores  (r  = 
.400,  t  =  7.23,  p  <  .001),  females’  higher  average  AFQT  scores  led  to  the  expectation  of  higher  than 
average  performance  on  the  firefighting  examination. 

A  regression  equation  was  developed  to  explore  female-male  differences  in  examination  performance 
adjusting  for  the  difference  in  general  mental  ability.  The  equation  was 

T'  =  3.613  +  (.00871  *  AFQT) 


where  T'  is  the  predicted  test  score. 

An  adjusted  test  score  (Tadj)  was  computed  to  measure  performance  controlling  for  ability.  This  score 
was  computed  by  subtracting  the  predicted  test  score  from  the  observed  test  score  (i.e.,  Tadj  =  T  -  T' 
where  T  is  the  test  score).  The  adjusted  scores  were  positive  if  performance  was  higher  than  expected 
given  the  person’s  ability  level,  and  negative  if  performance  was  lower  than  expected. 

b.  Treatment  Effects  on  Tadj 

Treatment  effects  on  academic  performance  were  evaluated  by  comparing  the  Tadj  scores  of  females  and 
males  within  each  treatment  group.  The  comparison  of  females  and  males  in  the  control  group 
supported  the  initial  claim  that  females  perform  relatively  poorly  in  this  component  of  basic  training. 
The  average  score  for  females  (mean  =  -.226)  was  lower  than  that  for  males  (.085)  as  expected. 
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Absent  any  treatment,  therefore,  the  female-male  difference  in  test  performance  was  d  =  -.311;  this 
difference  was  statistically  significant  (t  =  -3.24,  86  df,  p  <  .001,  one-tailed). 

The  RM  treatment  did  not  improve  females’  performance  on  the  firefighting  examination.  Females  who 
received  this  treatment  still  scored  below  average  after  adjusting  for  their  general  mental  ability  (mean 
=  -.240).  Males  who  received  this  treatment  had  scores  that  were,  on  the  average,  virtually  identical  to 
their  predicted  scores  (mean  =  .019).  The  difference  between  females  and  males  (d  =  -.259)  was  only 
slightly  smaller  than  that  under  the  control  conditions  (d  =  -.311).  Females  and  males  still  differed 
significantly  on  performance  in  this  treatment  (r  =  -2.64,  88  df,  p  <  .005,  one-tailed). 

The  AO  treatment  improved  females’  performance  on  the  firefighting  examination.  Females  in  this 
treatment  scored  only  slightly  below  the  level  expected  based  on  their  AFQT  scores  (mean  =  -.074). 
This  deficit  was  only  one-third  to  one-fourth  that  found  in  the  control  group  and  the  recruits  receiving 
the  RM  treatment.  Males  who  received  the  AO  treatment  scored  slightly  higher  (mean  =  .095)  than 
expected  based  on  their  AFQT  scores.  The  difference  between  females  and  males,  therefore,  remained 
moderately  large  (d  =  .169),  but  was  reduced  in  size  relative  to  that  in  other  treatments.  The  remaining 
female-male  difference  was  not  statistically  significant  (t  =  1.61,  85  df,  p  <  .056). 

The  AO  treatment  effect  is  placed  in  perspective  by  several  comparisons.  To  begin  with,  the  female- 
male  difference  in  this  treatment  was  only  about  one-half  that  in  the  control  group  (d  =  -.169  vs  d  = 
-.311).  The  trend  toward  a  smaller  gender  difference  for  the  AO  group  was  consistent  with  the 
hypothesis  that  this  treatment  would  improve  female  performance  relative  to  male  performance. 
However,  although  the  trend  was  in  the  predicted  direction,  it  was  only  marginally  significant 
(difference  =  .142,  t  =  1.34,  p  <  .092,  one-tailed). 

Comparing  the  two  treatments,  each  treatment  reduced  the  performance  deficit  in  females,  but  the  AO 
treatment  was  more  effective.  The  AO  treatment  cut  the  performance  deficit  for  female  recruits  by  46% 
compared  to  a  reduction  of  only  17%  for  RM.  Given  the  small  sample  sizes  in  the  present  study,  the 
AO  gain  was  only  marginally  significant,  but  the  RM  gain  did  not  even  approach  significance. 

c.  Treatment  Effects  on  Stress 

Treatment  effects  on  stress  were  estimated  using  anticipated  stress  as  a  covariate.  This  procedure  was 
used  instead  of  a  repeated  measures  analysis.  The  repeated  measures  analysis  assumes  that  the  stress 
scales  measured  the  identical  construct  at  both  the  pre-  and  post-training  administrations.  This 
assumption  was  questionable  on  logical  grounds  for  the  stress  and  efficacy  scales.  In  these  instances, 
the  post-training  scores  reflect  perceptions  and  feelings  based  on  actual  experiences  in  firefighting 
training.  Recruits  cannot  reasonably  be  expected  to  know  accurately  what  those  experiences  will  be 
prior  to  training.  Pre-test  variables,  therefore,  must  reflect  generalization  from  other  parts  of  basic 
training,  generalized  tendencies  to  perceive  stress,  trait  elements  of  mood,  or  other  similar  factors. 
Arguably,  the  measurements  taken  at  the  two  times  represent  distinct  constructs  (e.g.,  capacity  for 
stress  vs.  actual  stress),  in  which  case  a  difference  score  such  as  that  which  would  be  evaluated  in  a 
repeated  measures  design  is  difficult  to  interpret.  Using  the  pre-training  scores  as  covariates  of  the  post- 
training  scores  can  be  seen  as  a  method  of  adjusting  for  the  continuing  influence  of  those  factors  that 
determined  the  individual  differences  in  anticipation.  Removing  the  effects  of  those  factors  should  leave 
a  purer  indication  of  the  effects  of  training  on  stress. 
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Mood  scores  presumably  measured  the  same  constructs  at  both  testing  times.  These  scales,  therefore, 
could  reasonably  have  been  evaluated  using  a  repeated  measures  analysis  of  variance.  The  analysis  of 
covariance  was  employed  for  these  distress  scales  to  make  those  analyses  comparable  to  the  analyses  for 
the  other  stress  indices. 

Table  6  summarizes  the  findings  for  stress  as  a  function  of  gender  and  treatment  group  controlling  for 
anticipatory  stress.  The  only  significant  main  effects  were  those  obtained  for  gender  and  treatment  on 
role  conflict.  These  effects  were  present  because  females  experienced  less  conflict  than  males,  and  the 
Role  Model  treatment  was  associated  with  less  conflict.  The  estimated  mean  score  adjusted  for  pre¬ 
training  anticipatory  conflict  was  2.50  (SE  =  .08)  for  females  compared  to  2.74  (SE  =  .05)  for  males. 
The  corresponding  estimates  for  the  treatments  indicated  that  the  RM  treatment  had  the  lowest  value 
(mean  =  2.52,  SE  =  .08)  followed  by  the  control  group  (mean  =  2.65,  SE  =  .09)  and  the  AO 
treatment  group  (mean  =  2.69,  SE  =  .08).  The  primary  difference  between  treatment  groups  appeared 
to  be  the  contrast  between  the  extreme  groups.  The  95%  confidence  intervals  for  the  RM  treatment 
group  (2.36,  2.68)  did  not  include  the  mean  for  the  AO  group;  the  95%  confidence  interval  for  the  AO 
treatment  group  (2.54,  2.85)  did  not  include  the  mean  for  the  RM  treatment  group. 


Table  6.  Treatment  Effects  on  Stress  Indicators 


Scale 

Gender 

Treatment 

Gender  x  Treatment 

F 

Sig. 

F 

Sig. 

F 

Sig. 

Role  Ambiguity 

1.50 

.22 

0.03 

.973 

0.26 

.774 

Role  Conflict 

11.56 

.012 

1.22 

.298 

Self-Efficacy 

0.37 

.542 

2.07 

.128 

2.35 

.097 

Negative  Mood 

0.47 

.495 

0.72 

.487 

1.55 

.215 

Positive  Mood 

0.57 

.451 

0.66 

.518 

0.30 

.738 

d.  Extended  Analysis  of  Interactions 

Table  6  also  indicated  the  presence  of  one  marginally  significant  interaction  between  gender  and 
treatment.  This  marginal  effect  was  examined  in  more  detail  because  the  interaction  was  a  diffuse 
significance  test  (i.e.,  a  test  involving  more  than  one  degree  of  freedom;  cf.  Rosenthal  &  Rosnow,  1984). 
In  this  case,  one  or  more  meaningful  differences  might  be  concealed  because  they  were  mixed  with  a 
number  of  other  trivial  differences.  Given  that  this  phase  of  the  study  was  only  an  initial  test  of  the 
treatments,  exploration  of  this  possibility  was  reasonable. 

The  interactions  between  gender  and  treatment  approached  significance  in  the  omnibus  analyses  for  the 
adjusted  examination  score  and  self-efficacy.  Examination  of  the  average  scores  in  the  different 
treatments  suggested  that  this  circumstance  occurred  because  the  treatments  had  very  specific  effects  on 
females.  Despite  the  weakness  of  the  trends  for  these  differences,  further  evaluation  was  appropriate 
given  the  preliminary  character  of  the  study.  The  results  of  the  evaluation  will  direct  future  refinements  of 
the  treatments. 
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(1)  Self-Efficacy 


Examination  of  the  mean  scores  for  self-efficacy  suggested  that  the  RM  treatment  was  the  primary 
source  of  the  treatment  x  time  interaction  for  this  variable.  In  general,  females  and  males  had 
comparable  levels  of  anticipated  efficacy  in  firefighting  training.  Post-training  efficacy  generally  was 
comparable  for  females  and  males  adjusting  for  anticipated  efficacy.  However,  females  who  received 
the  RM  treatment  showed  exceptionally  high  self-efficacy  after  firefighting  training. 

An  adjusted  self-efficacy  post-training  score  was  computed.  Self-efficacy  was  prior  to  firefighting 
training  and  after  firefighting  training.  The  difference  between  the  predicted  and  observed  post- 
firefighting  training  scores  provided  the  adjusted  post-training  score.  Given  that  anticipated  self-efficacy 
quite  probably  reflected  a  generalized  self-efficacy  based  on  general  life  success  and  success  to  that  point 
in  basic  training,  the  adjusted  post-firefighting  training  self-efficacy  score  can  be  interpreted  as  an  index 
of  situation-specific  self-efficacy  scored  as  a  deviation  from  generalized  self-efficacy. 

Combining  the  AO  and  control  groups  into  a  single  group,  recruits  receiving  the  RM  condition  could  be 
compared  to  all  others.  The  resulting  2x2  ANOVA  indicated  that  the  RM  treatment  had  a  significant 
main  effect  (F1.273  =  4.18,  p  <  .042),  but  there  also  was  an  interaction  of  gender  with  treatment  status. 
The  average  adjusted  scores  indicated  that  treatment  status  had  little  average  effect  for  males  (RM  = 
.01;  Others  =  .02).  Among  females,  the  RM  treatment  was  associated  with  higher  adjusted  self- 
efficacy  (0.18).  Other  treatments  led  to  lower  self-efficacy  (-0.13).  The  difference  between  the  two 
groups  of  females  was  statistically  significant  (t  =  2.24,  p  <  .028)  and  translated  to  a  moderate  effect 
size  (ES  =  0.50). 

(2)  Negative  Mood 

The  existence  of  a  significant  effect  of  the  RM  treatment  on  one  psychological  stress  indicator 
motivated  a  search  for  other  indicators  with  a  similar  pattern.  Examination  of  the  average  pre-training 
and  post-training  scores  also  suggested  that  the  same  pattern  contrasting  the  RM  treatment  with  the  AO 
treatment  and  controls  was  present  for  negative  mood.  In  this  case,  the  interaction  effect  for  the  original 
2x3  ANOVA  was  not  even  marginally  significant  (p  <  .215;  cf.  Table  6),  and  even  the  interaction  for 
the  2  x  2  ANOVA  was  only  marginally  significant  ( Fi,26i  =  2.78,  p  <  .097). 

The  negative  mood  differences  were  of  interest  despite  their  weak  statistical  significance  because  they 
paralleled  the  self-efficacy  differences.  Once  again,  the  RM  treatment  had  little  effect  on  males  (RM  = 
.07;  Other  =  -.01)  and  a  positive  effect  on  females  (RM  mean  =  -.24;  Others  mean  =  .04).  The 
difference  between  females  receiving  the  RM  treatment  and  other  females  was  statistically  significant 
(t  =  1.76,  67  df,  p  <  .042,  one-tailed).  A  moderate  effect  size  (ES  =  .43)  was  evident. 

(3)  Examination  Performance 

Group  differences  in  examination  performance  were  explored  in  more  detail  using  Tadj  as  the  criterion 
measure.  The  group  means  from  the  gender  x  treatment  analysis  indicated  that  the  AO  improved  this 
outcome  for  females.  Females  who  received  the  AO  treatment  had  examination  scores  that  were  only 
slightly  less  than  the  score  that  would  be  predicted  based  on  their  AFQT  scores  (Avg.  =  -.07,  SD  = 
.39).  The  hypothesis  that  the  true  mean  score  for  females  receiving  this  treatment  is  zero  (i.e.,  that  the 
females  receiving  this  treatment  performed  up  to  the  level  expected  based  on  their  mental  ability) 
cannot  be  rejected  ( t  -  -0.82,  20  df,  p  <  .211,  one-tailed).  Females  who  did  not  receive  the  AO 
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treatment  scored  below  their  predicted  performance  (mean  =  -.23,  SD  =  .37,  n  =  40).  This  deficit 
was  significantly  below  zero  (r  =  3.93,  39  df,  p  <  .001).  The  performance  deficit  in  these  females  was 
lower  than  that  for  females  who  received  the  AO  treatment,  but  the  difference  was  only  marginally 
significant  (r  =  1.58,  p  <  .061). 

Comparing  females  to  males  who  received  the  same  treatment  also  illustrates  the  potential  value  of  the 
AO  for  improving  females’  academic  performance.  Among  those  receiving  no  treatment  (i.e.,  the  control 
condition),  males  performed  significantly  better  than  females  (male  mean  =  .09;  female  mean  =  -0.23;  t 
=  3.24,  86  df,  p  <  .001,  one-tailed).  The  same  trend  was  evident  in  the  RM  treatment  (males  =  0.02; 
females  =  -0.24;  t  =  2.64,  88  df,  p  <  .005,  one-tailed).  The  female-male  difference  was  substantially 
smaller  and  only  marginally  significant  statistically  in  the  AO  condition  (males  =  0.10;  females  =  -0.07;  t 
=  1.61,  p<. 056). 

Expressing  the  impact  of  the  AO  on  academic  performance  as  an  effect  size  provides  another  perspec¬ 
tive  on  the  effectiveness  of  this  treatment.  On  the  average,  the  adjusted  score  for  females  in  the  control 
and  RM  groups  was  0.29  points  lower  than  that  for  males  in  the  same  treatments.  The  comparable 
difference  was  0.17  between  female  and  male  recruits  receiving  the  AO  treatment.  Stated  this  way,  the 
AO  provided  an  improvement  of  0.12  points  relative  to  males  receiving  the  same  treatment.  This  gain 
represents  an  effect  size  of  ES  =  0.30  using  the  pooled  SD  for  all  males  as  the  denominator  for  the 
computation.  This  ES  value  is  near  the  middle  of  Cohen’s  (1988)  small  effect  size  range. 

4.  Individual  Differences  in  Stress  and  Performance 

The  analyses  described  thus  far  examined  group  differences  in  stress  and  performance.  Those  analyses 
compared  average  scores  for  different  groups,  and  treated  individual  variation  within  groups  as  error 
variance.  This  approach  did  not  test  for  possible  associations  between  individual  differences  in  stress 
and  individual  differences  in  performance.  Given  that  it  is  individuals  who  actually  perceived  stress  and 
performed  on  the  examination,  the  grouping  process  may  have  been  somewhat  misleading.  Additional 
analyses,  therefore,  were  undertaken  to  examine  associations  between  stress  and  performance  with  the 
individual  as  the  unit  of  observation. 

The  additional  analyses  evaluated  the  hypothesis  that  stress  affects  performance  by  Tadj  to  adjusted  stress 
scores.  The  adjusted  scores  corrected  post-training  stress  and  distress  reports  for  the  effects  of  pre¬ 
training  anticipatory  stress  and  distress.  Table  7  shows  the  results  of  these  analyses  for  the  full  sample, 
all  females,  all  males,  all  participants  in  the  control  group,  all  participants  in  the  AO  treatment,  and  all 
participants  in  the  RM  treatment. 

a.  Full  Sample 

Anticipatory  stress  was  not  related  to  Tadj  (r  =  .007  to  r  =  .066).  Adjusted  post-training  scores 
generally  were  unrelated  to  Tadj,  but  good  performance  was  associated  with  a  larger  than  expected 
decrease  in  negative  mood  (r  =  -.14,  p  <  .024). 
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Table  7.  Adjusted  Stress  Scores  as  Predictors  of  Adjusted  Academic  Performance 


Scale 

Total 

Sample 

Females 

Males 

Control 

Advance 

Organizer 

Role 

Model 

Role  Ambiguity 

.006 

.035 

.011 

-.023 

.037 

.008 

Role  Conflict 

-.134 

-.047 

.001 

-.053 

Self-Efficacy 

-.007 

-.136 

.032 

.117 

Negative  Mood 

-.144* 

.016 

-.208** 

-.323** 

-.094 

-.242* 

Positive  Mood 

-.067 

.149 

00 

O 

f 

-.046 

-.130 

.008 

*p  <  .05  **p  <  .01 


Note:  Sample  sizes  varied  slightly  depending  on  the  amount  of  missing  data  for  the  different  measures. 
The  ranges  of  sample  sizes  were:  Total  Sample  =  247-259;  Females  =  58-61;  Males  =  189-198; 
Control  =  82-86;  Advance  Organizer  =  82-85;  Role  Model  =  83-88. 


b.  Gender 

It  was  thought  that  gender  might  affect  the  associations  between  stress  and  performance.  However, 
gender  comparisons  produced  ambiguous  results.  First,  the  Tadj-negative  mood  association  was  limited 
to  males  (r  =  -.21).  This  association  was  essentially  zero  among  females  (r  =  .02).  However,  the 
difference  between  the  correlations  was  only  marginally  significant  statistically  (z <as  =  1.48,  p  <  .070) 
using  the  Fisher  r-to-z  transformation  (Hays,  1963,  p.  532).  This  result  was  ambiguous  because  a 
simple  “Is  the  association  significant  in  both  groups?”  test  for  differences  would  lead  to  the  inference 
that  the  groups  differed  while  asking  “Are  the  two  correlations  significantly  different?”  which  would 
lead  to  the  inference  that  the  females  and  males  did  not  differ. 

The  results  for  a  second  correlation  were  ambiguous  because  they  demonstrated  the  opposite  pattern. 
Among  females,  adjusted  role  conflict  was  positively  related  to  academic  performance  (r  =  .178). 
Among  males,  the  correlation  was  negative  (r  =  -.134).  The  difference  between  these  correlations  was 
statistically  significant  (zm  —  2.20,  p  <  .014),  but  neither  of  the  individual  correlations  was 
significant.  Thus,  the  data  indicated  a  significant  difference  from  one  perspective  while  making  it 
impossible  to  rule  out  the  following  equality:  T females  —  finales  —  .00. 

The  fact  that  the  determination  of  statistical  significance  is  dependent  on  the  establishment  of  a 
significance  criterion  adds  to  the  ambiguity  of  the  role  conflict-Tadj  correlations  for  females  and  males. 
The  observed  difference  was  labeled  “significant”  using  the  common  p  <  .05  criterion.  However,  role 
conflict  was  one  of  five  potential  predictors  of  academic  performance  considered  in  the  analysis. 
Conservative  significance  testing  would  apply  a  Bonferroni  adjustment  to  maintain  the  experiment-wide 
error  at  p  <  .05  (Harris,  1985,  pp.  7-9).  In  this  case,  the  significance  criterion  would  move  to  p  < 
.01;  the  observed  result  would  be  classified  as  nonsignificant  using  that  criterion.  Given  these  additional 
considerations,  the  evidence  for  female-male  differences  in  stress-test  performance  relationships  is 
equivocal. 
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c.  Treatment 


Treatments  also  could  affect  stress-performance  relationships.  This  possibility  received  some  support 
from  comparisons  for  negative  mood.  Adjusted  negative  mood  was  significantly  related  to  adjusted 
academic  performance  in  the  control  (r  =  -.323,  n  =  82,  p  <  .003,  two-tailed)  and  RM  (r  =  -.242, 
n  =  83,  p  <  .028,  two-tailed)  treatment  groups,  but  not  the  AO  group  (r  =  -.094,  n  =  82,  p  < 
.401).  The  differences  appear  to  be  substantial,  but  they  are  not  statistically  significant.  Comparing  the 
AO  treatment  to  the  control  group,  the  difference  approached  significance  (z<*y  =  1.51,  p  <  .132,  two- 
tailed).  The  difference  between  the  AO  treatment  and  the  RM  treatment  did  not  even  approach 
significance  (zw  -  0.96,  p  <  .338,  two-tailed).  Two-tailed  significance  tests  were  reasonable  in  these 
comparisons  because  there  were  no  a  priori  hypotheses  about  the  location  and  direction  of  the 
differences  tested.  These  results  were  ambiguous  in  the  same  way  that  the  gender  difference  in  the 
negative  mood-Tadj  relationship  was  ambiguous. 
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III.  DISCUSSION 


The  initial  evaluation  of  the  new  instructional  technologies  supports  several  general  conclusions.  The 
results  confirm  that  females  experience  more  difficulty  than  males  in  learning  the  classroom  material  in 
firefighting  training.  The  results  indicated  that  the  AO  treatment  reduced  the  relative  deficit  in  academic 
performance,  but  the  RM  treatment  did  not.  In  contrast,  the  results  indicated  that  the  RM  treatment  had 
some  positive  effects  on  distress  and  efficacy,  but  the  AO  treatment  did  not. 

The  evidence  provided  no  reason  to  believe  that  stress  was  a  mediator  of  academic  performance,  but  no 
inference  could  be  drawn  about  stress  and  leadership  or  team  performance  because  suitable  criteria 
were  not  available.  Finally,  preliminary  descriptions  of  the  patterns  of  individual  interactions  with  the 
AO  treatment  were  identified  that  may  provide  a  basis  for  improving  the  treatment.  Each  of  these  points 
is  considered  in  more  detail  in  the  following  paragraphs. 

A.  PERFORMANCE  OF  FEMALES 

Females  did  relatively  poorly  on  the  academic  test.  The  control  group  provides  the  best  estimate  of  this 
deficiency  as  it  currently  exists  in  training  because  these  females  essentially  received  the  same  treatment 
that  recruits  in  general  receive  at  present.  The  average  adjusted  score  for  the  females  in  the  control 
group  was  -.23.  This  figure  was  0.58  standard  deviation  below  the  average  adjusted  score  for  males. 
The  difference  represents  a  moderately  large  effect  size  in  behavioral  research  (Cohen,  1988).  This 
evidence  quantifies  the  size  of  the  problem  being  addressed  with  greater  precision  than  was  possible  at 
the  study  outset. 

B.  EFFECTS  OF  TREATMENT 

1.  Advance  Organizer  Treatment  Effect 

The  AO  treatment  improved  the  academic  performance  of  female  recruits.  A  slight  performance 
decrement  relative  to  ability  still  was  evident  in  this  group  (mean  =  -.07),  but  the  deficiency  was  less 
than  one-third  the  size  of  that  noted  in  both  the  control  and  RM  treatment  groups.  Furthermore, 
statistical  tests  indicated  that  the  observed  AO  deficiency  was  not  significantly  different  from  zero  or 
significantly  different  from  that  of  males  going  through  the  same  treatment.  The  overall  pattern  of 
findings  indicates  that  the  AO  treatment  reduces  the  female  academic  performance  deficiency  and  may 
eliminate  it  altogether. 

2.  Role  Model  Treatment  Effect 

The  RM  treatment  did  not  affect  academic  performance,  but  this  result  was  not  surprising.  The  RM 
treatment  was  designed  to  improve  performance  in  the  hands-on  portion  of  firefighting  training.  The 
emphasis  on  teamwork  and  leadership  was  expected  to  be  rewarding  when  recruits  had  to  communicate 
and  work  together  to  put  out  fires.  The  project  was  undertaken  with  the  belief  that  suitable  hands-on 
firefighting  measures  would  be  available  to  assess  individual  differences  in  performance.  These 
measures  were  not  developed.  The  improved  sense  of  efficacy  and  better  mood  at  the  end  of  training 
suggest  that  female  recruits  who  went  through  the  RM  treatment  felt  they  did  better.  Recruits’ 
spontaneous  comments  (see  Appendix  A)  also  suggested  they  felt  this  was  a  useful  treatment.  Thus,  it 
seems  likely  that  RM  affected  performance,  but  not  in  the  areas  that  could  be  directly  measured. 
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3.  Hawthorne  and  Placebo  Effects 


The  fact  that  the  RM  and  AO  treatments  affected  different  outcomes  is  important.  The  AO  treatment 
affected  academic  performance,  while  the  RM  treatment  affected  stress.  This  difference  helps  rule  out  a 
Hawthorne  effect  (Shaughnessy  &  Zechmeister,  1985,  pp.  336-338)  or  placebo  effect  as  the  explanation 
for  the  treatment  effects.  A  Hawthorne  effect  would  occur  if  simply  giving  some  people  more  attention 
resulted  in  improved  performance.  A  placebo  effect  would  occur  if  a  treatment  effect  was  present  only 
because  the  recruits  receiving  the  treatment  believed  it  would  affect  them.  In  the  present  case,  there  was 
no  reason  for  recruits  to  believe  that  one  treatment  would  be  useful,  but  the  other  would  not.  Given 
these  assumptions,  both  groups  received  comparable  attention  and  received  treatments  that  they 
believed  would  be  efficacious.  Neither  assertion  applied  to  the  control  group.  The  result  is  that  the 
placebo  and  Hawthorne  effect  interpretations  both  would  differ  from  the  control  group,  but  would  be 
comparable  to  one  another.  A  pattern  of  differential  effects  for  the  AO  and  RM  treatments  is  contrary 
to  these  expectations. 

4.  Differential  Treatment  Effects 

The  differential  effects  for  the  AO  and  RM  treatments  become  more  plausible  when  it  is  noted  that 
these  effects  occur  where  they  might  be  expected  to  occur,  based  on  the  nature  of  the  treatments.  The 
AO  treatment  was  designed  to  prepare  a  cognitive  basis  for  learning  academic  materials.  The  RM 
treatment  illustrates  patterns  of  interpersonal  interaction  and  behaviors  that  are  basic  to  role 
communication  and  role  enactment  and,  therefore,  are  basic  to  the  occurrence  of  role  ambiguity  and 
role  conflict.  The  AO  treatment  effect  may  be  the  result  of  changes  in  the  cognitive  processes  involved 
in  learning  the  academic  material  in  firefighting.  The  RM  treatment  may  modify  the  stress  perceptions 
and  other  psychological  reactions  to  firefighting  training  that  develop  out  of  the  teamwork  component  of 
training.  Taken  together,  these  considerations  help  rule  out  nonspecific  causal  explanations  (e.g., 
Hawthorne  and  placebo  effects)  in  favor  of  causal  explanations  based  on  the  specific  content  of  the 
treatments.  The  pattern  of  results,  therefore,  provides  a  basis  for  inferring  that  the  treatments  did  in  fact 
cause  the  differential  performance  and  stress  effects  observed  in  this  study. 

5.  Stress  Effect 

Stress  reduction  was  not  associated  with  improved  academic  performance.  This  inference  is  based  on 
the  lack  of  connection  between  stress  and  performance.  First,  the  AO  treatment  improved  females’ 
academic  performance,  but  did  not  reduce  stress.  Second,  the  RM  treatment  reduced  some  stresses,  but 
did  not  improve  academic  performance.  Third,  individual  differences  in  stress  were  not  related  to 
individual  differences  in  performance.  The  only  suggestion  of  a  relationship  was  that  negative  mood 
and  Tadj  were  significantly  related.  However,  that  association  could  be  explained  as  well  by  assuming 
that  poor  academic  performance  led  to  increased  depression,  anger,  and  so  on.  Also,  even  the 
association  between  negative  mood  and  Tadj  was  absent  in  the  one  treatment  that  actually  modified 
academic  performance  (i.e.,  the  Advance  Organizer  treatment).  Exploratory  analyses  suggested  that  the 
relationships  between  stress  and  academic  performance  might  differ  for  males  and  females  or  between 
different  treatment  groups,  but  chance  sampling  variation  could  not  be  completely  ruled  out  as  a  basis 
for  those  differences. 

The  RM  treatment  reduced  stress  among  females.  Females  receiving  the  RM  treatment  showed  gains  in 
self-efficacy  and  reduced  negative  mood  at  the  end  of  firefighting  training.  Because  males  and  females 
were  comparable  on  these  two  measures  at  the  beginning  of  firefighting  training,  these  changes  mean  that 
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females  in  the  RM  treatment  stood  out  as  being  lower  on  these  stress  indicators  relative  to  all  other  males 
and  females. 

Females  and  males  generally  reported  comparable  stress  levels.  The  only  significant  difference  between 
genders  was  for  females  to  report  lower  role  conflict  after  firefighting  training.  Given  evidence  that 
females  possess  personality  characteristics  associated  with  reporting  more  negative  affect  (Feingold, 
1994),  females  might  have  been  expected  to  have  higher  negative  mood  and  perhaps  lower  positive 
mood.  However,  in  this  setting,  the  difference  was  not  evident. 

The  finding  that  females  and  males  experience  comparable  stress  levels  is  consistent  with  prior  research 
(Martocchio  &  O’Leary,  1989).  Other  reviewers  have  suggested  that  females  report  more  affective 
stress  than  males  (Jick  &  Mitz,  1985).  However,  the  effect  sizes  involved  were  not  reported  in  that 
review,  and  the  sample  sizes  for  the  majority  of  the  studies  were  quite  large  (n  >  700).  This  evidence 
may  point  to  a  general  tendency  toward  higher  affective  stress  in  females,  but  the  difference  could  be 
trivial.  Furthermore,  the  recruit  situation  is  different  from  the  general  population  in  that  males  and 
females  have  the  same  roles,  face  the  same  living  situations,  etc.  Even  if  female-male  differences  in 
affective  stress  are  present  in  the  general  population,  the  differences  might  disappear  if  the  two  groups 
were  equated  for  occupational  status,  education,  family  situation,  and  other  factors  as  they  are,  to  a 
great  extent,  in  basic  training.  A  general  belief  that  females  are  more  susceptible  to  stress  than  males 
may  exist.  If  so,  based  on  the  results  of  this  study,  that  belief  appears  to  be  a  misleading  stereotype. 

C.  LIMITATIONS  OF  PRESENT  STUDY 

Sample  size  and  contamination  were  two  main  limitations  in  the  present  study.  Research  during  Year 
Three  of  the  study  will  address  these  limitations. 

1.  Sample  Size 

Initial  findings  are  based  on  small  sample  sizes  that  provided  the  intended  preliminary  evaluation  of  the 
treatments.  The  analyses  frequently  explored  marginally  significant  trends  in  the  data  without  allowing 
for  the  extent  to  which  these  explorations  could  capitalize  on  chance.  This  approach  was  appropriate  for 
a  preliminary  evaluation  that  was  intended  to  provide  information  useful  for  refining  the  study 
hypotheses.  Strict  adherence  to  traditional  significance  testing  criteria  combined  with  broad  general 
hypotheses  might  have  resulted  in  important  effects  being  overlooked  by  using  diffuse  significance 
tests.  However,  the  approach  taken  here  can  be  interpreted  only  as  establishing  a  set  of  refined  working 
hypotheses  for  further  study.  The  findings  need  to  be  replicated  with  larger  sample  sizes  and  drawing 
on  recruits  who  go  through  training  at  different  times  of  the  year.  The  latter  requirement  arises  because 
important  recruit  qualities  change  over  the  course  of  the  year.  The  effectiveness  of  the  treatments  could 
depend  on  one  or  more  attributes  of  recruits.  It  is  impossible  to  say  a  priori  whether  the  recruit 
differences  will  increase  or  decrease  the  effectiveness  of  the  treatments. 

The  sample  size  in  the  initial  study  was  also  too  small  to  study  the  effects  of  the  individual  differences 
in  learning  style.  Such  differences  may  impact  the  effectiveness  of  the  AO  treatment.  Preliminary 
evidence  was  gathered  this  year  (see  Appendix  D  for  data  tables)  that  indicates  the  existence  of 
individual  differences  in  how  people  interact  with  the  computer  in  this  condition  (e.g.,  choosing  the 
linear  approach  versus  the  concept  map,  the  number  of  questions  asked,  the  types  of  questions  asked, 
etc.).  These  differences  may  influence  how  well  the  recruits  learn  firefighting  material.  If  so,  studying 
the  effects  of  the  differences  on  performance  could  provide  helpful  clues  for  refining  the  AO  treatment 
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to  improve  its  overall  effectiveness.  General  principles  relevant  to  all  types  of  technical  training  might 
be  identified,  but  it  is  too  early  to  tell.  This  problem  will  be  overcome  by  increasing  the  sample  size, 
quantifying  differences  in  the  pattern  of  interaction  with  the  computer,  and  relating  those  differences  to 
performance  outcomes. 

2.  Contamination 

Some  contamination  may  have  occurred  in  the  original  study  design  because  of  shared  interest  on  the 
part  of  recruits  and  instructors.  “ Contamination  occurs  when  there  is  communication  of  information 
about  the  experiment  between  groups  of  subjects”  (Shaughnessy  &  Zechmeister,  1985,  p.  335,  italics  in 
the  original).  Each  division  studied  included  recruits  who  were  exposed  to  the  AO,  RM,  and  control 
treatments.  Recruits  in  the  control  group  knew  that  others  were  receiving  the  experimental  treatments. 
Instructors  were  concerned  that  materials  learned  in  the  treatments  would  help  only  some  of  the  recruits 
in  their  division  when  all  of  the  recruits  could  potentially  benefit.  Anecdotal  evidence  indicates  that 
exposure  to  the  treatments  stimulated  recruits  to  change  their  behaviors  in  several  ways,  including 
asking  more  questions  in  later  parts  of  classroom  training  and  participating  more  actively  in 
discussions.  These  behaviors  changed  the  nature  of  subsequent  learning  opportunities  not  only  for  the 
recruits  within  a  particular  treatment,  but  also  for  all  the  other  recruits  in  their  division.  This  type  of 
contamination  may  have  acted  to  influence  treatment  effectiveness.  The  potential  contamination 
problems  will  be  minimized  during  next  year’s  data  collection  by  changing  the  research  design  so  that  a 
single  treatment  will  be  administered  to  all  recruits  within  a  division  or  within  paired  divisions. 
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IV.  RECOMMENDATIONS 


There  are  four  main  recommendations  for  future  research,  as  described  in  the  following  paragraphs. 

•  Focus  on  Advance  Organizer  Treatment 

The  AO  treatment  will  be  the  primary  focus  of  the  next  year’s  work.  This  treatment  demonstrated  a 
sizable  effect  on  academic  performance,  a  meaningful  criterion  for  training  personnel.  That 
demonstration  needs  to  be  replicated.  Preliminary  evidence  was  obtained  indicating  that  different 
people  interact  differently  with  this  treatment.  The  small  sample  size  makes  it  impossible  to  say 
whether  differences  in  the  interaction  pattern  affect  the  outcome  from  exposure  to  this  treatment. 
The  sample  size  needs  to  be  increased  for  this  treatment  in  order  to  permit  more  detailed  analysis  of 
the  relationship  between  patterns  of  interaction  and  treatment  effectiveness. 

Replication  of  the  RM  treatment  will  not  be  implemented.  While  the  RM  treatment  appeared  to  be 
effective,  the  effects  on  behavioral  variables  cannot  be  demonstrated  in  the  current  firefighting 
training  context.  For  this  reason,  it  is  not  possible  to  directly  demonstrate  that  the  treatment  has  a 
positive  effect  on  outcomes  that  are  central  objectives  of  basic  training.  Reduced  stress  may  be 
good,  but  it  is  valuable  in  a  training  context  primarily  if  it  demonstrates  a  positive  effect  in 
achieving  learning  objectives.  The  stress  effects  provide  reason  to  believe  that  the  RM  treatment 
can  affect  training  outcome  criteria,  but  sound  behavioral  assessments  of  leadership  and/or  team 
effectiveness  would  be  required  to  extend  this  work.  Such  measures  could  be  developed,  but  even 
then  the  highly  structured,  carefully  controlled  setting  of  firefighting  training  might  be  a  poor  place 
to  examine  these  effects.  Recruit  comments  suggest  that  other  situations,  such  as  Service  Week, 
provide  more  extensive  leadership  and  team  performance  opportunities.  Some  recruits  commented 
that  they  felt  this  treatment  would  have  helped  them  during  that  time  period.  However,  developing 
this  line  of  inquiry  is  beyond  the  scope  of  available  resources  for  this  project.  The  RM  treatment 
should  be  given  consideration  for  further  research  if  more  resources  become  available,  but  until 
then,  the  benefit  seems  greater  for  focusing  on  the  AO  treatment. 

•  Reduce  Contamination 

The  research  design  will  be  modified  to  eliminate  possible  sources  of  treatment  contamination.  The 
initial  design  was  guided,  in  part,  by  a  desire  to  gather  useful  information  on  both  AO  and  RM 
treatments  with  as  little  intrusion  into  training  as  possible.  Based  on  the  initial  findings,  we  will 
implement  a  revised  design  that  exposes  all  of  the  recruits  in  a  division  to  a  single  treatment.  The 
control  group  will  be  comprised  of  recruits  from  divisions  who  go  through  the  regular  training 
schedule  with  no  treatment. 

•  Evaluate  Individual  Differences  in  Learning  Style 

Research  during  Year  Three  of  the  study  will  capitalize  on  the  opportunity  to  examine  individual 
differences  in  learning  style.  Demographic,  academic  and  educational  variables  will  also  be 
assessed  for  their  role  on  how  academic  data  is  assimilated  and  how  it  impacts  the  curricular 
successes  of  male  and  female  recruits.  Students  interact  with  the  AO  treatment  in  a  manner  that  is 
consistent  with  their  individual  learning  styles.  AO  interaction  is  defined  as  selection  of  method 
(concept  map  or  linear)  and  selection  of  questions  within  the  AO  treatment.  By  evaluating 
individual  differences  in  learning  styles,  we  will  gain  insight  into  which  strategies  are  the  most 
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successful  at  preparing  students  of  different  genders,  ethnicity,  education,  intelligence,  and  age  for 
the  academic  portion  of  the  course. 

Two  more  data  collections  will  be  conducted.  The  first  will  provide  information  to  better  quantify 
individual  differences  in  patterns  of  interaction  with  the  AO.  This  data  will  increase  the  sample  size 
sufficiently  to  estimate  the  influence  of  these  patterns  on  academic  proficiency.  The  second  study 
will  replicate  those  findings  and  might  involve  some  modifications  to  the  AO  treatment  itself.  The 
timing  of  the  two  data  collections  will  be  chosen  to  verify  that  the  AO  effect  is  present  for  recruits 
who  enter  the  service  at  different  times  of  year. 

Evaluate  Gender  Differences  in  Stress-Performance  Associations 

Stress  assessments  will  be  retained  as  part  of  the  Year  Three  design.  Although  the  overall  evidence 
from  this  study  suggests  that  stress  and  performance  were  not  generally  related,  the  comparison 
between  the  AO  and  control  groups  appears  to  be  the  most  likely  place  to  find  an  exception  to  this 
generalization.  Among  controls,  poor  performance  was  associated  with  more  negative  mood  at  the 
end  of  firefighting  training.  Receiving  the  AO  treatment  appeared  to  reduce  this  effect,  but  the 
trend  was  only  marginally  significant.  Replication  of  the  initial  finding  is  needed  to  determine 
whether  this  difference  indicates  an  effect  of  AO.  Also,  the  evidence  suggested  that  females  and 
males  differ  with  regard  to  stress-performance  relationships.  Perhaps  this  difference  exists  primarily 
in  specific  treatment  conditions.  The  sample  sizes  in  the  present  study  were  too  small  to  evaluate 
gender  differences  in  stress-performance  associations  within  specific  treatments.  Increasing  the 
sample  size  for  the  control  and  AO  groups  will  permit  more  detailed  analysis  of  this  possibility. 
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V.  CONCLUSIONS 


The  work  conducted  during  the  past  year  included  implementation  of  two  instructional  treatments  and 
an  initial  evaluation  of  their  effectiveness.  In  summary,  the  initial  results  indicate  that  the  three  null 
hypotheses  postulated  for  this  study  were  disproved. 

•  First  Null  Hypothesis 

Providing  women  with  models  of  effective  performance  will  not  reduce  stress  and  will  not  improve 
job  performance. 

Initial  findings  are  contrary  to  this  hypothesis.  The  results  indicate  that  both  the  AO  and  RM 
treatments  have  positive  effects,  but  the  two  treatments  produce  effects  in  different  areas.  The  RM 
treatment  had  some  positive  effects  on  distress  and  efficacy.  Female  recruits  were  found  to  feel 
better  about  themselves  and  had  a  decrease  in  negative  mood.  The  RM  treatment  focused  on  the 
behaviors  required  in  actual  firefighting,  and  these  behaviors  and  the  related  group  dynamics  are 
fundamental  to  the  psychological  constructs  of  role  ambiguity  and  role  conflict.  The  RM  treatment 
fostered  self-efficacy  by  providing  models  of  people  who  succeeded  in  firefighting  training. 

•  Second  Null  Hypothesis 

Providing  women  with  aids  to  structure  the  learning  process  will  not  reduce  stress  and  will  not 
improve  job  performance. 

Initial  findings  disproved  this  hypothesis.  The  AO  treatment  did  improve  learning  for  females  by 
providing  a  structure  for  acquiring  and  storing  technical  material.  This  treatment  is  linked  to  stress 
only  indirectly  in  that  stress  is  reduced  when  greater  knowledge  reduces  role  ambiguity  and  role 
conflict  and  improves  the  person’s  sense  of  self-efficacy.  However,  stress  apparently  is  not 
important  in  the  processes  linking  the  AO  treatment  directly  to  improved  learning.  While  female 
recruits  were  projected  to  have  higher  scores  in  academic  tests  because  of  their  higher  Armed 
Forces  Qualification  Test  (AFQT)  levels,  they  scored  lower  than  their  male  counterparts.  Academic 
performance  deficiencies  for  females  who  completed  the  Advance  Organizer  were  reduced.  Male 
scores  also  improved  for  the  AO  participants,  albeit  not  as  dramatically  as  the  scores  of  the 
females. 

•  Third  Null  Hypothesis 

Providing  men  and  women  with  models  of  effective  performance  will  not  result  in  male/female 
attitudinal  changes. 

This  hypothesis  could  not  be  fully  explored  during  Year  Two.  The  stress  indicators  suggested  that 
the  Role  Model  intervention  was  found  to  have  a  positive  effect  on  female  efficacy  and  negative 
mood.  However,  the  effect  of  role  modeling  on  the  attitude  of  male  recruits  toward  female  recruits 
could  not  be  assessed  because  a  suitable  behavioral  criterion  representing  individual  and/or  team 
effectiveness  in  the  hands-on  element  of  firefighting  training  was  not  available.  These  results  imply 
that  the  RM  treatment  holds  promise  of  being  an  effective  procedure  given  an  appropriate  criterion. 
The  corresponding  team  performance  criteria  and  measurement  necessary  to  test  this  hypothesis 
were  not  developed  at  the  RTC  because  of  procedural  and  resource  constraints.  / 
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The  initial  findings  must  be  interpreted  cautiously,  given  the  small  sample  size  and  the  somewhat 
exploratory  character  of  the  data  analysis.  The  findings  clearly  indicate  that  the  AO  treatment  merits 
further  analysis  and  refinement.  Replication  of  the  study  and  an  in-depth  analysis  of  the  Advance 
Organizer  will  provide  insight  into  strategies  used  by  the  recruits  to  synthesize  technical  material. 

Demographic,  academic,  and  educational  variables  will  be  assessed  for  their  role  on  how  academic  data 
is  assimilated  and  how  it  impacts  the  curricular  successes  of  the  male  and  female  recruits.  Specifically, 
the  variables  we  will  be  assessing  are:  gender,  formal  education,  ethnicity,  age,  general  intelligence  as 
measured  by  scores  on  AFQT,  and  attendance  at  remedial  instruction  received  in  the  FAST  lab.  We 
will  compare  each  of  these  variables  with  the  academic  post-test  scores,  the  computer-based  test  scores, 
and  AO  interaction.  AO  interaction  is  defined  as  selection  of  method  (concept  map  or  linear)  and 
selection  of  questions  within  the  AO  treatment.  This  information  will  provide  us  with  insight  into  which 
strategies  are  the  most  successful  at  preparing  students  of  different  genders,  ethnicity,  education, 
intelligence,  and  age  for  the  academic  portion  of  the  course. 

The  intense  scrutiny  of  recruit  training  by  Congress  and  the  media  with  regard  to  gender  integration  has 
forced  the  Navy  to  re-evaluate  its  training  pedagogy.  The  data  yielded  by  this  study  will  prove 
invaluable.  Training  policy  decisions  and  instructional  system  design  strategies  will  have  a  sound  base 
built  on  concrete  data.  Additionally,  as  the  demographics  of  the  recruit  population  continue  to  evolve  to 
include  a  greater  percentage  of  females,  minorities,  and  older  recruits,  the  Navy  will  have  data  with 
which  to  develop  effective  remedial  programs  that  embed  sophisticated  instructional  technology  such  as 
intelligent  tutors.  Finally,  the  results  of  this  study  can  be  generalized  and  will  be  of  functional  use  to 
our  national  educational  communities. 
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APPENDIX  A 

SPECIFIC  COMMENTS  FROM  DATA  COLLECTION  EFFORT 


Date 

Comp 

Program 

Program 

Location/Subj 

Commen¬ 

tator 

Comment 

6/15/98 

Gen 

AO 

Animation 

Eductor  animation  does  not  play. 

6/15/98 

Gen 

AO 

Animation 

Miller 

Only  2  recruits  from  this  lab 
mentioned  that  animation  was 
missing.  Suggests  that  no  one  else 
hit  that  page.  If  they  were  in  the 
linear,  they  went  to  the  test  early. 

If  in  the  map,  they  either 
intentionally  went  to  the  test  early 
or  didn’t  realize  they  had  not  hit 
that  map. 

6/15/98 

14 

AO 

CM 

Miller 

No  background  image.  (Reloaded 
program  after  class  and  could  not 
reproduce  problem.) 

6/19/98 

45 

AO 

CM 

Goodman 

Recruit  missed  sections  because  he 
could  not  tell  which  maps  he  had 
hit  and  which  he  had  not. 

6/19/98 

46 

AO 

CM 

Recruit 

“Are  we  supposed  to  take  test  now 
or  end  of  sections  or  when?”  Recruit 
confused  because  of  screen  (?). 

6/15/98 

Gen 

Condition 

Miller 

Recruits  from  this  lab  are  in  very 
poor  condition  -  very  tired;  falling 
asleep;  eyes  are  bloodshot. 

6/16/98 

Gen 

RM 

Condition 

Goodman 

Some  nodding  off  during 
testimonials  and  video  info.  Awake 
for  interactive  questions. 

6/16/98 

Gen 

Condition 

Miller/ 

Goodman 

Lab  temperature  much  warmer. 

May  have  affect  on  more  frequent 
nodding. 

6/19/98 

Gen 

Condition 

Goodman 

Much  cooler.  AC  working  much 
better. 

RM 

Condition 

Goodman 

Recruits  are  fidgety  and  not  “into” 
testimonials. 

6/19/98 

Gen 

AO/RM 

Condition 

Goodman 

2  or  3  recruits  at  any  given  time 
nodding  off  within  15  minutes  of 
starting. 

6/15/98 

27 

RM 

Condition 

Goodman 

Recruit  nodding  off  regularly 
during  video  clips  -  awake  only  to 
push  forward  and  answer  questions. 

6/15/98 

33 

RM 

i  Condition 

Goodman 

Heavy  nodding/sleeping  throughout. 

6/15/98 

41 

RM 

1 

Condition 

Goodman 

Recruit  nodding  off  regularly 
during  video  clips  -  awake  only  to 
push  forward  and  answer  questions. 

6/15/98 

Gen 

Condition 

Goodman 

Glazed  eyes  during  info  videos; 

When  interacting  with  video  and 
questions,  looked  more  lively. 

Commen¬ 

tator 


Comment 


„  Program 

rogram  Location/Subj 


Condition  Miller 


Extinguishing  Goodman 
Systems 


Feedback  Recruit 


O  Feedback  Recruit 


Feedback 


Feedback 


Feedback 


Feedback 


RM  Feedback 


Recruit  to 
Miller 


Recruit 


Recruit 


Recruit 


Recruit 


26  6/16/98  Gen  RM/AO  Feedback 


Goodman 


Recruits  were  more  alert.  RDC  (2n 
sitting-females)  stayed  in  the  lab  but 
this  did  not  appear  to  intimidate  the 
students.  Students  were  relaxed  and 
comfortable  but  did  not  sleep  to  the 
degree  noted  in  the  first  night. 

RDC  (1st  sitting  -  males)  had  been 
“pumping  up”  firefighting  to  his 
recruits  -  telling  of  his  personal 
experiences  with  shipboard  fires,  the 
importance  of  firefighting,  etc. 


Did  not  erase  pics  behind  question 
box.  Box  and  examples. 


“Got  a  lot  of  studying  to  do.” 
(Recruit  wrote  down  topics 
suggested  for  review  from  Test 
Summary  page.) 


“What  is  the  difference  between 
‘displacing  oxygen’  and  ‘removing’ 
oxygen.”  (Made  Miller  aware  that 
program  is  not  consistent  with 
verbiage.  Program  should  be 
consistent  with  4  ways  of  putting 
out  a  fire. 


Miller  noticed  that  recruit  did  not 
finish  test.  When  asked  about  it, 
recruit  said  program  “just  quit”  after 
Q4.  Said  he  did  not  hit  the  Quit 
button.  Recruit  added  “I  got  nothing 
out  of  this.  Sound  was  no  good.  Get 
rid  of  the  voice.”  Recruit  said  he 
would  get  more  out  of  the  program 
with  the  data  directly  on  the  screen. 


Need  a  warning  (highlight  button)  of 
when  to  go  to  next  page  by  forward 
button.  She  suggested  audio  cue. 


“Good  program.” 


“How  can  I  hear  the  question 
again?”  (Suggested  adding  a 
‘replay’  button  for  test  questions  (?) 


“That’s  a  lot  of  information.” 


“Very  good  program.  Let  me  see 
gear  that  we’ll  be  using  and  the 
activities  in  FF.  Should  have  people 
watch  before  Service  Week  so  they 
understand  chain  of  command  -  like 
in  galley.  Really  good  -  aH  need  to 
go  through!” 


Received  several  enthusiastic  thank- 
yous  from  recruits  on  the  way  out  of 
the  lab. 


No.  Date 


Program  Commen- 

Location/Subj  tator 


27  6/16/98  Gen 


Feedback 


44  6/15/98  Gen 


Chief 

Campbell  to 

Goodman/ 

Miller 


Chief  Campbell  had  noticed  that  the 
students  that  day  in  the  “Basic  Fire 
Fighting”  class  had  asked  “more 
specific  related”  questions.  “Basic 
Fire  Fighting”  is  the  first  class  in  the 
firefighting  training.  The  Chief  was 
impressed  assuming  that  the 
students  had  “read  ahead”  not 
realizing  that  his  class  had  been  the 
first  of  the  recruits  to  be  a  part  of 
our  study. 


28 

6/15/98 

Gen 

RM 

Feedback 

Recruits 

(several) 

Noticed  instructors  they’d  had  and 
seen  “at  this  base”. 

29 

6/15/98 

Gen 

NA 

Forward 

Miller 

30 

6/19/98 

13 

AO 

Frozen 

Goodman 

Quit  was  frozen  on  last  screen.  Had 
to  ctrl-alt-del  to  end  task  to  get  out  - 
Data  ok? 

31 

6/19/98 

19 

AO 

Frozen 

Goodman 

Quit  was  frozen  on  last  screen.  Had 
to  ctrl-alt-del  to  end  task  to  get  out  - 

Data  ok? 


User  function  code  error.  When 
recruit  makes  initial  choice,  pop-up 
box  appears  and  gives  error  message 
instead  of  “Are  you  sure. . 

Reloaded  computer  twice. 


Fielded  questions  about  which  route 
to  take  (i.e.,  linear  or  map?) 
throughout  the  week. 


Recruit  smiling  and  snickering 
during  testimonials. 


Laughed  when  got  comments  and 
viewing  both  sailors’  testimonials. 


Several  recruits  laughing  and 
smiling  at  videos. 


Snickering  and  smiling  through 
interactive  questions  and  recruit 
videos. 


Bravo  fire  -  Remember  question  - 
Text  out  of  align. 


Section  titles  do  not  show  u 


“Do  I  just  go  to  any  of  them?” 


“If  there  are  watertight 
compartments,  why  did  the  Titanic 
sink?” 


Staman,  506-13-9016.  Disk  out  for 
sign  on  -  punched  in  before  “ok” 
button. 


Recruit  says  she  went  through  the 
whole  program  but  screen  does  not 
reflect  gray. 


Appeared  to  have  a  good  split  of 
students  choosing  linear  or  map. 
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No.  Date 


45  6/15/98  Gen 


46  6/15/98  Gen 


47  6/15/98  05 


48  6/16/98  05 


59  6/19/98  20 


60  6/19/98  20 


61  6/19/98  21 


62  6/19/98  21 


63  6/19/98  44 


64  6/15/98  Gen 


65  6/15/98  Gen 


66  6/19/98  Gen 


Program 

Location/Subj 

Commen¬ 

tator 

Comment 

Misc 

Miller 

Recruits  tend  to  choose  “Poor 
shipmate”  on  all  questions. 

Misc 

Goodman 

Did  not  observe  anyone  pause/play 
(buttons)  video.  Maybe  relabel  as  . 
“REPLAY”. 

Notes 

Recruit 

“Can  I  write  this  stuff  down?” 

Notes 

Miller 

Recruit  took  his  time.  Took  notes. 
Compare  test  score. 

Notes 

Goodman 

Used  Training  Guide  -  filling  in 
blanks.  Took  notes.  In  close 
proximity  to  20  and  21 . 

Notes 

Goodman 

Notetaker.  Refers  to  notes  on  Q17 
and  system  Qs. 

Notes 

Goodman 

Notetaker.  Refers  to  notes  on  Q17 
and  system  Qs. 

Notes 

Miller 

Recruit  took  his  time.  Took  notes. 
Compare  test  score. 

Notes 

Goodman 

Notetaker.  Refers  to  notes  on  Q17 
and  system  Qs. 

Notes 

Miller 

Recruit  took  his  time.  Took  notes. 
Compare  test  score. 

Notes 

Miller 

Had  to  request  recruit  to  begin  the 
test.  Notetaker. 

Notes 

Miller 

Recruit  took  his  time.  Took  notes. 
Compare  test  score. 

Notes  i 

Goodman 

Notetaker.  Refers  to  notes  on  Q17 
and  system  Qs. 

Notes 

Goodman 

Used  Training  Guide  -  filling  in 
blanks.  Took  notes.  In  close 
proximity  to  7  and  21. 

Notes 

Goodman 

Used  notes  and  Training  Guide 
while  taking  test. 

Notes 

Goodman 

Notetaker.  Refers  to  notes  on  Q17 
and  system  Qs. 

Notes 

Goodman 

Used  Training  Guide  -  filling  in 
blanks.  Took  notes.  In  close 
proximity  to  7  and  20. 

Notes 

Goodman 

Notetaker.  Refers  to  notes  on  Q17 
and  system  Qs. 

Notes 

Goodman 

Notetaker.  Refers  to  notes  on  Q17 
and  system  Qs. 

Notes 

Recruit 

“Can  I  take  notes?”  (Recruit  went  to 
the  question  button  several  times. 
Remember  style.) 

Notes 

Miller 

Fielded  several  “Can  I  take  notes?  / 
Should  I  take  notes?”  type  of 
questions. 

Notes 

Goodman 

Fielded  questions  such  as  “Is  this 
info  on  the  final  test?”,  “Should  I  be 
taking  notes?”  and  “Will  I  see  this 
again?” 
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Date 


Commen¬ 

tator 


Comment 


Comp  Program  ^Sbj 


67  6/16/98  01  AO  Overs 


69 

6/16/98 

70 

6/19/98 

71 

6/16/98 

2  6/16/98  16  AO  Overs 


AO  Overs 


6/16/98 

Gen 

6/16/98 

Gen 

6/15/98 

Gen 

6/16/98 

Gen 

6/19/98 

Gen 

Question 


Question 


Question 


Quit 


Miller/ 

Goodman 


Miller/ 

Goodman 


Miller 


Miller 


Recruit 


Goodman 


Goodman 


Goodman 


Recruits 


Goodman 


6/16/98  -  Recruit  started  over 
because  he  wanted  to  go  through  the 
program  again.  Collected  only  the 
data  information  from  his  first  try. 


6/16/98  -  Recruit  started  over 
because  he  wanted  to  go  through  the 
program  again.  Collected  only  the 
data  information  from  his  first  try. 


6/16/98  -  Recruit  started  over 
because  he  wanted  to  go  through  the 
program  again.  Collected  only  the 
data  information  from  his  first  try. 


Restarted  V2  way  through  lessons. 
He  had  double-clicked  sign-on 
twice.  So  2  AOs  running.  2nd  AO 
came  to  front  somehow.  Quit  out  of 
1st  and  continue  in  the  2nd.  Recruits 
start  more  than  one  sign-on  often. 


Recruit  started  program  over  on  his 
own.  When  asked  why,  he  replied 
“I  hit  the  wrong  ke 


Recruit  quit  the  program  and 
restarted  on  his  own.  (Take  second 
session  on  data  disk.) 


6/16/98  -  Recruit  started  over 
because  he  wanted  to  go  through  the 
program  again.  Collected  only  the 
data  information  from  his  first  try. 


Notice  recruit  used  the  question 
button  several  times. 


Section  9  -  Question  erase  button. 


What  is  the  Question  Mark  button 
for?  Oh,  ok.  now  how  do  I  get  back? 


Kill  last  screen.  Quit  off  menu 
straight  out  of  program. 


RDC  pumped  up  guys  to  pay 
attention  to  this  info  because  it  saves 
lives  -  “most  important  info  in  boot 
camp.” 


RDC  stayed  in  room.  Motivated 
(nicely)  people  to  stay  awake. 


Wanted  to  click  in  more  than  1  place 
for  problems  (storage  shelves) 


How  do  I  pick  male/female?  Need 
to  change  from  mouse  click  to  key 
stroke. 


Several  asked  how  to  go  back  to 
entry  before  getting  to  the  Correct 
button. 


No. 

Date 

Comp 

Program 

Program 

Location/Subj 

Commen¬ 

tator 

Comment 

83 

6/15/98 

Gen 

Sound 

Miller 

Sound  really  low.  Chief  Ennis 
indicated  that  only  a  few  machines 
had  good  quality  sound  cards.  We 
purchased  higher  quality  headsets 
but  this  only  slightly  improved 
sound. 

84 

RM 

Teamwork 

Recruit 

Did  Teamwork  topic  twice  because 
did  not  get  all  of  it. 

85 

6/19/98 

24 

RM 

Teamwork 

Goodman 

Teamwork  exercises  have  no  titles. 

86 

6/19/98 

43 

RM 

Teamwork 

Goodman 

Doesn’t  have  any  audio  -  only  on 
this  one.  Teamwork  -  team  being 
lethargic. 

87 

6/15/98 

Gen 

RM 

Teamwork 

Goodman 

No  audio  on  Teamwork  video  with 
team  being  lethargic.  Recruit  (user) 
thinks  something  is  wrong. 

88 

6/16/98 

05 

AO 

Test 

Miller 

Test  score:  70%.  Same  topics 
highlighted  as  16.  Since  computers 

5  and  16  are  not  close  to  each  other, 
check  data  to  compare  which  actual 
questions  missed.  May  be  the  way 
question  is  worded. 

89 

6/19/98 

08 

AO 

Test 

Recruit 

“I  got  to  the  test  by  accident.”  (Was 
on  the  last  screen  so  saw  all  of  the 
questions.)  He  chose  map  then  hit 
the  wrong  button  by  accident. 

90 

6/16/98 

16 

AO 

Test 

Miller 

Test  score:  70%.  Same  topics 
highlighted  as  5.  Since  computers 

5  and  16  are  not  close  to  each  other, 
check  data  to  compare  which  actual 
questions  missed.  May  be  the  way 
question  is  worded. 

91 

6/15/98 

18 

AO 

Test 

Miller 

Pop-up  box  -  Can’t  find  9q2  video. 
(All  of  Section  9  videos  are  missing 
from  directory  on  computer. 

Reloaded  computer  after  class.) 

92 

6/15/98 

Gen 

AO 

Test 

Recruit 

Well  - 1  failed  that  test!  Can  I  do 
this  program  again?  How  can  I 
remember  this? 

93 

6/15/98 

Gen 

AO 

Test 

Miller 

Test  title  font  wrong. 

94 

6/15/98 

Gen 

AO 

Test 

Miller 

No  topic  buttons  (?) 

95 

6/16/98 

Gen 

AO 

Test 

Miller 

Several  recruits  asked  if  they  were 
to  take  the  test  after  each  section. 

96 

6/16/98 

Gen 

AO 

Test 

Miller 

Question  about  small  Alpha  fire, 
notice  many  recruits  wanting  to 
click  both  Water  and  C02. 

97 

6/19/98 

Gen 

AO 

Test 

Goodman 

Noticed  several  unsure  and  asked 
me  about  how  many  answers  can 
give  on  test  -  for  example  1,  11,  12, 
etc. 

98 

6/15/98 

03 

AO 

Test-Q12 

Font  in  answers  (Q12  &  Q13)  too 
big  (?).  User  could  drag  and  drop  so 
answers  did  not  line  up. 

99 

6/15/98 

Gen 

AO 

Test-Q12 

Miller 

Make  Q12  pictures  non-moveable. 

39 


102  6/16/98  07 


103  6/16/98  11 


104  6/15/98  12 


105  6/16/98  12 


Program 

Location/Subj 

Commen¬ 

tator 

Comment 

Test  -  Q13 

Recruit 

Skipped  Q13.  Went  from  Q12  to 

Q14. 

Test-Q13 

Miller 

Program  did  not  lock  up  at  Q13. 
Recruit  does  not  remember  if  he  got 
it  right/wrong  or  even  if  Q13 
showed  up.  Check  data  disk. 

Test-Q13 

Miller 

Skipped  Q13  and  Q 14.  Went  from 
Q12  to  Q15.Q12  wrong.  (2nd 
sitting.) 

Miller 

Skipped  Q13.  Went  from  Q12  to 

Q14.  Q12  wrong.  (2nd  sitting.) 

Test-Q13 

Miller 

Skipped  Q13.  Went  from  Q12  to 

Q14. 

Test-Q13 

Miller 

Skipped  Q13.  Went  from  Q12  to 

Q14.  Q12  wrong.  (2nd  sitting.) 

Test-Q13 

Miller 

Q13  freezes  if  answer  is  wrong. 

Can’t  quit,  escape.  Have  to 
completely  restart.  (Went  through 
linear.) 

Test  -  Q13 

Miller 

Skipped  Q13.  Went  from  Q12  to 

Q14.  Q12  wrong.  (2nd  sitting.) 

Test  -  Q13 

Miller 

Q13  freezes  if  answer  is  wrong. 

Can’t  quit,  escape.  Have  to 
completely  restart.  (Went  through 
map.) 

Test-Q13 

Miller 

Q13  freezes  if  answer  is  wrong. 

Can’t  quit,  escape.  Have  to 
completely  restart. 

Test-Q13 

Goodman 

Lost  data.  Froze  when  Q13  wrong. 
Could  not  quit  and  disk  was  out  so 
no  info. 

Test  -  Q13 

Miller 

Did  not  notice  that  any  computers 
skipped  Q1 3  in  1st  sitting.  Asked 
that  recruits  raise  their  hand  at  Q12 
and  we  followed  them  through  Q13 
and  gave  the  correct  answer  to  avoid 
program  from  locking  up. 

Test-Q17 

Miller 

Q17  replays  (?)  Stuck  in  a  loop. 
Program  will  not  go  to  Q18. 

Problem  appears  to  be  limited  to  this 
computer. 

Test  -  Q17 

Miller 

Need  better  instructions  on  Q17. 
Recruits  do  not  appear  to  understand 
what  is  expected  of  them  or  they 
only  drag  1  picture  for  each  column 
rather  than  all  the  pictures  into  the 
correct  column. 

Test  -  Q3 

Miller 

Can’t  find  video  for  Q3.  Other 
computers  do  not  appear  to  be 
missing  video.  Assume  this  is  a 
problem  limited  to  this  computer. 

Test  -  Q3 

Miller 

Q3  videos  -  recruit  dragged  videos 
rather  than  titles. 

40 


No. 

Date 

Comp 

Program 

Program 

Location/Subj 

Commen¬ 

tator 

Comment 

i  16 

6/19/98 

Test  -  Q3 

Goodman 

Q3  -  if  drop  names  behind  videos, 
can’t  get  to  them  again.  Background 
moves.  Memory  low  -  can’t  play 
videos. 

117 

6/15/98 

Gen 

AO 

Test  -  Q3 

Goodman 

Can  drag  videos  around  on  Question 

3  -  Ship’s  Bells  and  General  Alarm. 

118 

6/15/98 

Gen 

AO 

Test  -  Q3 

Miller 

Question  3  -  bells  and  general 
alarm.  Confused  about  instructions. 
Recruit  dragged  the  video. 

119 

6/16/98 

Gen 

AO 

Test  -  Q5 

Miller 

Q5  -  “How  can  I  replay  the 
question?”  “What  condition  did  the 
question  say?” 

120 

6/15/98 

29 

RM 

Testimonials 

Goodman 

Laughed  when  got  comments  and 
viewing  both  sailors’  testimonials. 

121 

6/15/98 

38 

RM 

Testimonials 

Recruit 

There  isn’t  a  “both”  choice  on  the 
screen  -  so  where  click? 

122 

6/19/98 

Testimonials 

Goodman 

Testimonials  need  titles. 

123 

6/15/98 

03 

AO 

Time 

Miller 

Began  test  at  30  minutes.  (Indicates 
went  straight  through  -  no 
questions/topics,  etc.) 

124 

6/15/98 

03 

AO 

Time 

recruit 

“I  thought  I  had  to  take  the  test  to 
get  out  of  section.” 

Went  to  test  at  5  minutes. 

125  j 

6/15/98 

06 

AO 

Time 

Miller 

Began  test  at  30  minutes.  (Indicates 
went  straight  through  -  no 
questions/topics,  etc.) 

126 

6/19/98 

07 

AO 

Time 

Goodman 

Recruit  stayed  until  time  up  to 
continue  going  through  information 
and  taking  notes. 

127 

6/15/98 

20 

AO 

Time 

Miller 

Began  test  at  30  minutes.  (Indicates 
went  straight  through  -  no 
questions/topics,  etc.) 

128 

6/16/98 

Time 

Goodman 

3  recruits  finished  at  30  minutes.  3 
more  at  35  minutes.  All  but  2 
finished  @  40  minutes.  Last  2  had 
been  sleeping/taking  breaks 
throughout. 

129 

6/16/98 

Gen 

RM 

Time 

Goodman 

All  recruits  done  between  30  and  45 
minutes. 

130 

6/19/98 

05 

AO 

Video 

Goodman 

Video  8-4  missing.  (Check  that  path 
shows  ./src/video.) 

131 

6/16/98 

16 

AO 

Video 

Miller 

Check  Q2-5  video  (?). 

132 

6/15/98 

14 

AO 

Video  8-4 

Miller 

Video  8-4  missing.  (Check  that  path 
shows  ./src/video.) 

133 

6/16/98 

Gen 

AO 

Video  8-4 

Miller 

Video  8-4  missing.  (Check  that  path 
shows  ./src/video.) 

134 

6/16/98 

13 

AO 

Videos 

Miller 

Tutorial  videos  missing.  Computer 
not  program  (reload  after  class). 

CM  -  Concept  Map 

NA  -  Not  Applicable 

R DC  -  Recruit  Division  Commander 
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APPENDIX  B 

CHANGES  MADE  TO  ADVANCE  ORGANIZER 
AND  ROLE  MODEL  INTERVENTIONS 
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APPENDIX  B 


CHANGES  MADE  TO  ADVANCE  ORGANIZER 
AND  ROLE  MODEL  INTERVENTIONS 


Change 

Video  3-1  doesn’t  play,  corrected. 

Eductor  animation  doesn’t  play,  corrected. 

Extinguishing  systems  -  did  not  erase  pics  behind  question  box, 
corrected. 

Correct  inconsistent  text  problem  (displace/remove  fuel,  etc.). 

Correct  Question  13  lock-up  problem.  Program  skips  it  or  locks  up. 

Check  font  on  Q13,  corrected. 

Check  font  on  Q12,  corrected. 

Make  pics  on  Q12  non  movable. 

Bravo  fire,  remember  question,  text  out  of  alignment,  corrected. 

Section  Titles  (AO  linear)  do  not  show  up,  corrected. 

Teamwork  seems  to  skip  sections,  corrected. 

Test  title  font  wrong,  corrected.’ 

Make  pics  on  Q3  nonmovable. 

Q3  -  background  moves,  corrected. 

Q3  -  if  name  drops  behind  video,  can't  retrieve  it  to  finish  question, 
corrected. 

Re-record  parts  of  script  (correct  forward/next  issue,  etc.). 

Redo  Graphical  User  Interface  to  make  it  more  readable. 

Reconcile  “poor  shipmate”  problem  by  making  it  “as  applied  to  this 
situation”. 

Did  not  observe  any  recruits  pausing  or  replaying  video.  Relabel  button 
as  “Replay”. 

Add  method  of  leaving  questions  without  asking  a  question. 

Delete  last  screen  so  can’t  Quit  off  menu  straight  out  of  program  (RM). 

Change  Male/Female  from  mouse  click  to  key  stroke. 

Team  lethargic  -  RM  -  redo  audio. 

Teamwork  exercises  have  no  titles,  corrected. 

Provide  better  instructions  on  Q17.  Students  want  to  drag  only  one  for 
each  subject. 

Testimonials  need  titles,  corrected. 

Q3,  video  of  bell,  make  video  end  at  bell. 

Add  more  text  instructions  throughout. 

Show  section  complete  on  map. 

Highlight  “Everything”  and  “Specific”  buttons  as  they  are  discussed. 

Add  cues  for  forward. 
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APPENDIX  C 

SCREEN  DISPLAYS  FROM  ADVANCE  ORGANIZER 
AND  ROLE  MODEL  TREATMENTS 
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To  begin 


5.  What  exactly  do  I  need  to  remember? 


QUIT 


Press  anykey  to  t> 


BACK  i  FORWARD 


SADERSHIP 


BACK  I  FORWARD 


BACK  9  FOR’ 


APPENDIX  D 

ADMINISTRATIVE  AND  INTERVENTION 
DATA  TABLES 
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APPENDIX  D 

ADMINISTRATIVE  AND  INTERVENTION  DATA  TABLES 

The  administrative  data  and  data  concerning  the  students’  interactions  within  the  interventions  are 
presented  in  the  seven  tables  in  this  appendix.  Six  of  the  tables  are  included  at  the  end  of  this  appendix 
and  described  in  paragraphs  1  through  6  below.  The  seventh  table  is  included  in  paragraph  7  of  this 
appendix. 

1.  Administrative  Data 

The  Administrative  Data  table  was  used  as  an  organizational  tool  to  determine  which  recruits  had 
completed  all  the  necessary  paperwork  in  order  to  include  their  data  in  the  study.  For  an  individual 
recruit,  the  following  information  is  presented  in  the  table: 

•  Social  Security  Number  (SSAN) 

•  First  and  Last  Name 

•  Gender 

•  Division  Number 

•  Experimental  Group 

•  Advance  Organizer,  A 

•  Role  Model,  R 

•  Control,  C 

•  Consent  form  completion 

•  Completion  of  Stress  Profile  questionnaire  (pre/post-firefighting  training) 

•  Exam  score 

2.  Pre-Stress 

The  Pre-Stress  table  organizes  the  data  gathered  during  the  Stress  Profile  questionnaire  given  BEFORE 
the  treatments  and  firefighting  training.  For  an  individual  recruit  the  following  information  is  presented: 

•  SSAN 

•  Experimental  Group  (Program) 

•  Advance  Organizer,  AO 

•  Role  Model,  RM 

•  Control,  C 

•  Gender 

•  Division  Number 

•  Responses  to  Stress  Profile  questionnaire  for  all  40  questions  (Q1-Q40)  on  a  scale  of  1  to  5  as  related 
to  the  survey 

3.  Post-Stress 

The  Post-Stress  table  is  similar  to  the  Pre-Stress  table  but  organizes  the  responses  of  the  Stress  Profile 
questionnaire  given  AFTER  the  treatments,  firefighting  training,  and  firefighting  exam  (exam  #4).  The 
same  information  is  present  but  the  responses  are  identified  by  PQ1-PQ40. 
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4.  Sdata 


The  Sdata  table  was  generated  from  within  the  Advance  Organizer.  The  following  information  was 

collected  for  each  recruit  (91  total)  who  interacted  with  the  treatment. 

•  SSAN 

•  Session  Time  -  amount  of  total  time  spent  in  the  Advance  Organizer  including  the  test. 

•  First_Selected  -  indicates  the  route  the  recruit  chose  to  move  through  the  AO. 

•  Tutorial  -  linear  tutorial  method 

•  Concept  -  concept/lesson  map  method 

•  Second_Selected  -  indicates  the  recruit’s  second  choice  in  the  program,  which  is  always  the  test. 

•  Third_Selected  -  this  variable  was  incorporated  during  early  planning  stages  to  track  when  the 
student  moved  between  linear,  concept,  or  test.  In  the  current  design  it  is  not  used. 

•  Role  -  sum  total  of  the  number  of  role-related  questions  (possible  8  maximum)  a  recruit  clicked  on 
while  moving  through  the  system.  Answers  to  role  questions  consisted  of  video,  photo  stills,  and 
audio  elements.  Questions  are  similar  to  “Why  is  it  important  for  me  to  know  this?”  If  the  number  is  2 
or  greater,  the  recruit  was  automatically  fed  role-related  information  for  the  remainder  of  the 
treatment. 

•  Remember  -  sum  total  of  the  number  of  remember-related  questions  (possible  10  maximum)  a  recruit 
clicked  on  while  moving  through  the  system.  Answers  to  the  remember  questions  consisted  of  a  text 
and  audio  summary  of  the  main  points  of  a  topic/chapter.  If  the  number  is  2  or  greater,  the  recruit  was 
automatically  fed  role-related  information  for  the  remainder  of  the  treatment. 

•  Gender  -  sum  total  of  the  number  of  gender-related  questions  (possible  5  maximum)  a  recruit  clicked 
on  while  moving  through  the  system.  Answers  to  gender  questions  consisted  of  video,  photo  stills, 
and  audio.  Example  questions  are  “How  heavy  is  the  firefighting  equipment?”  and  “Is  it  harder  for 
females  to  serve  as  a  firefighter?”  If  the  number  is  2  or  greater,  the  recruit  was  automatically  fed  role- 
related  information  for  the  remainder  of  the  treatment. 

•  LSQ_1  -  LSQ_42  -  Learning  Style  Questions  (LSQ)  were  available  for  the  recruit  to  choose  on  all  of 
the  instructional  screens  throughout  the  Advance  Organizer.  If  a  recruit  chose  a  question,  the  table 
will  show  a  “1 .”  The  LSQs  are  listed  on  the  following  page. 

5.  Stest 

The  Stest  table  was  also  generated  by  the  Advance  Organizer.  It  holds  all  of  the  AO  test  data. 

•  SSAN 

•  Test_Score  -  There  are  20  questions,  each  worth  5  points  for  a  total  possible  score  of  100. 

•  Q1-Q20  -  Indicates  whether  the  recruit  got  the  question  correct  (1)  or  incorrect  (0).  Questions  that 
required  more  than  one  response  (questions  3,  9,  and  17)  have  more  than  one  column  to  record  the 
data  for  each  part  of  the  question.  The  answer  to  a  multiple-part  question  had  to  be  entirely  correct  for 
the  student  to  receive  the  5  points  in  the  scoring. 
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LEARNING  STYLE  QUESTIONS 


Chapter 

LSQ# 

Question  Type 

Question 

1 

LSQ_1 

Role 

Why  is  it  important  for  me  to  know  this  information? 

1 

LSQ_2 

Like 

How  is  this  like  something  I  already  know? 

1 

LSQ_3 

Remember 

What  exactly  do  I  need  to  remember? 

2 

LSQ_4 

How  do  I  call  the  officer  of  the  deck? 

2 

LSQ_5 

Like 

How  is  this  like  something  I  already  know? 

2 

LSQ_6 

Topic  specific 

How  do  I  help  the  fire  team  when  they  arrive? 

2 

LSQ_7 

Role 

Why  is  it  important  for  me  to  know  this? 

2 

Remember 

What  exactly  do  I  need  to  remember  ? 

3 

Who  sounds  the  alarms? 

3 

LSQ_10 

Like 

How  is  this  like  something  I  already  know? 

3 

LSQ_1 1 

Role 

I’m  not  a  firefighter.  Why  is  it  important  for  me  to  know 
this? 

3 

LSQ_12 

Remember 

What  exactly  do  I  need  to  remember? 

4 

LSQ.13 

Like 

How  is  this  like  something  I  already  know? 

4 

LSQ_14 

Gender 

Do  women  serve  on  all  ships? 

4 

LSQ_15 

Can  you  explain  the  identification  system  further? 

4 

LSQ_16 

Role 

Why  is  it  important  for  me  to  remember  this  numbering 
system? 

4 

LSQ_17 

Remember 

What  exactly  do  I  need  to  remember? 

5 

LSQ_18 

When  is  this  information  used  in  daily  ship  life? 

5 

LSQ.19 

Gender 

Who  closes  the  fittings? 

5 

LSQ_20 

Topic  specific 

What  happens  when  an  emergency  situation  is  over? 

5 

LSQ_2 1 

Role 

Why  is  it  important  for  me  to  know  this? 

5 

LSQ_22 

Remember 

What  exactly  do  I  need  to  remember? 

6 

LSQ_23 

Where  is  the  device  located? 

6 

LSQ_24 

How  does  the  device  work? 

6 

LSQ_25 

Role 

Why  is  it  important  for  me  to  know  this? 

6 

LSQ_26 

Remember 

What  exactly  do  I  need  to  remember? 

7 

LSQ_27 

How  heavy  is  all  the  protective  clothing? 

7 

LSQ_28  • 

BEESBEE1S3B 

How  long  does  it  take  to  put  on  all  the  equipment? 

7 

LSQ_29 

How  many  fires  occur  on  board  a  ship? 

7 

Remember 

What  exactly  do  I  need  to  remember? 

8 

Gender 

Is  it  harder  for  females  to  perform  as  a  firefighter? 

8 

LSQ_32 

Role 

I  am  not  a  DC-man.  Why  is  it  important  for  me  to  know 
this? 

8" 

LSQ_33 

Like 

How  is  this  information  like  something  I  already  know? 

8 

LSQ.34 

Remember 

What  exactly  do  I  need  to  remember? 

9 

LSQ_35 

Role 

I’m  not  a  DC-man.  Why  is  this  information  important  for 
me  to  know? 

9 

LSQ_36 

Gender 

Do  females  have  problems  using  any  of  these  systems 
because  of  the  equipment  weight? 

9 

LSQ_37 

Topic  specific 

What  happens  if  I  use  the  wrong  extinguishing  agent  or 
system? 

9 

LSQ_38 

Topic  specific 

After  recruit  training,  will  I  be  further  trained  on  these 
systems? 

9 

LSQ_39 

Remember 

What  exactly  do  I  need  to  remember? 

10 

LSQ_40 

Gender 

How  heavy  are  the  pumps  and  eductors? 

10 

LSQ_41 

Topic  specific 

What  are  examples  of  how  the  different  eductors  are  used? 

10 

LSQ_42 

Remember 

What  exactly  do  I  need  to  remember? 
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6.  Srole 


The  Srole  table  was  generated  by  the  Role  Model  treatment.  It  consists  of  all  recruit  role  model  data 

gathered  during  the  evaluation. 

•  SSAN 

•  Session  Time  -  the  entire  time  a  recruit  interacted  with  the  Role  Model  treatment. 

•  First_selected  -  the  topic  first  selected  (out  of  four  possibilities)  by  the  recruit  from  the  main  menu. 

•  Second_selected  -  the  topic  selected  second  by  the  recruit.  A  recruit  was  able  to  exit  the  treatment  at 
any  time.  If  there  is  a  “NONE”  entered  in  the  table,  the  recruit  chose  to  exit  the  program  before 
entering  all  of  the  possible  topic  areas  (Teamwork,  Leadership,  Safety,  Communication). 

•  Third_selected  -  the  topic  selected  third  by  the  recruit. 

•  Fourth_selected  -  the  topic  selected  fourth  by  the  recruit. 

•  El_3_male,  El_3_female,  El_3_both  -  the  E  stands  for  exercise.  The  numbers  (#_#)  refer  to  the 
storyboard  frame  numbers  that  describe  a  specific  exercise  or  testimonial.  Columns  that  are  similar  to 
these  labels  refer  to  the  “testimonials”  by  the  navy  sailors.  A  male  and  female  spoke  on  each  of  the 
four  topics.  If  a  recruit  clicked  on  one  of  them  to  hear  their  comments,  a  “1”  is  entered  in  the  column. 
If  a  recruit  chose  to  hear  both  the  male  and  female  testimonials,  a  “1”  is  entered  in  the  appropriate 
column. 

•  E2_10_Student  Response,  E2_10_Correct  -  the  E  stands  for  exercise.  The  numbers  (#_#)  refer  to  the 
'  storyboard  frame  numbers  used  during  the  development  process.  The  student  response  is  the  recruit’s 

answer  to  the  question.  The  correct  column  indicates  whether  the  student  got  the  correct  answer  (1). 

•  E2_12_l_a,b,c,d  -  the  E  stands  for  exercise.  The  numbers  once  again  refer  to  the  storyboard  frame 
number.  The  a,  b,  c,  d  refer  to  the  possible  answers.  If  a  recruit  chose  one  of  these  a  “true”  is 
indicated  in  the  table. 

7.  Recruit  Participant  and  Time  Data 

The  following  tables  show  the  number  of  recruits  participating  in  the  study  by  division  and  the  amount  of 

time  in  minutes  it  took  the  recruits  to  complete  the  interventions. 


NUMBER  OF  NON-PARTICIPATING  RECRUITS 


Division  # 

Total# 

Recruits 

Participated 

Difference* 

233  M 

76 

58 

18 

234  F 

42 

34 

8 

235  M 

67 

48 

19 

236  F 

46 

36 

10 

237  M 

56 

42 

14 

238  M 

62 

53 

9 

TOTALS 

349 

271 

78 

*Difference  is  due  to  extenuating  circumstances  as  described  in  Section  II.A.4, 
“Procedures  for  Implementation  of  Interventions  and  Data  Collection,”  or  to 
voluntary  non-consent. 
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SESSION  TIME  DESCRIPTIONS 
(Minutes) 

Role  Model 

Time  Interval  #  Recruits 

20-29  3 

30-39 _ 52 

40-49 _ 24 

50-59  10 


Time  (min) 

Recruit  Average 

39 

Recruit  Median 

38 

40 

Female  Median 

37 

39 

Male  Median 

38 

Advance  Organizer 

Time  Interval 

#  Recruits 

10-19 

3 

20-29 

0 

30-39 

12 

40-49 

40 

50-59 

21 

60-69 

7 

70-79 

5 

80-89 

2 

90 

1 

Total 

91 

Time  (min) 

49 

Recruit  Median 

47 

46 

Female  Median 

45 

50 

Male  Median 

50 

76 


Administration  Records 


8/27/98 


SSAN 


02781065 

29641195 


42703652 


65728767 


88662527 

137865156 


FirstName 


IJosh _ 

IKEVIN 


IDANIEL 


iCHRISTOPHE 


IChristopher 

jIAN 


27272077 

IKEVIN 

27418850 

IJEREMY 

30410090 

Brian 

33316233 

iRYAN 

35355563 

IJOSHUA 

40556411 

lAndre 

45041869 

IERIC 

LastName 


[Gardner 

[DILLON 


INOVAK 


IMATTE 


|Wolfert 


(HELWIG 


MCGOWAN 


DUNN 


Minnick 


OUTRIGHT 


DEMPSEY 


Patterson 


Gender  I  Drv  #  I  Group  I  Consent  |  Pre  SSQ  iPost  SSQl  Exam 


16989493 

20805335 

28685245 


jAaron 

jMARCUS 

'ROBERTO 


>30681212 

Manuel 

30724123 

QUYNTEN 

60669734 

GREG 

61745659 

Robert 

63925421 

Derek 

96869815  1 

BRIAN 

29671807 

James  • 

85 


34698831 


37211754 


50957390 

52634332 

57657206 

63796866 


83903771 


87946653 


15766180 


17119820 


517863523 


20889977 


25677055 


27933369 


30965411 


40928468 


3176102 


543257262 


DARYL 


IJEFFREY 


RONALD 


KENNETH 


Jonathan 


Bradley 

Timothy 

JAMES 

LARRY 

SCOTT 


Matthew 

GEORGE 


AARON 

VINCENT 


NICHOLAS 

Nicholas 


Joshua 


Andrew 


5578423 

ANTHONY 

JOHNSON 

h-  O 


Administration  Records 


8/27/98 


SSAN 

I  FirstName 

I  LastName 

I  Gender 

1  Div  #  |  Group  |  Consent 

im^in mmawmsm 

553794902 

iTERENCE 

(JUERGENS 

IMale 

j  233IR 

IX 

IX 

IX 

4.8 

572613258 

PATRICK 

ICURNYN 

IMale 

233lA 

X 

!x 

ix 

13.2 

575821013 

IMichael 

(O'Donnell 

IMale 

233C 

X 

lx 

ix 

13.8 

576672514 

IDORLITO 

ISENSANO 

IMale 

233  R 

X 

ix 

IX 

601645120 

IBRADLEY 

(MURREN 

IMale 

233  R 

X 

x 

ix 

14.4 

602886386 

IEUGENE 

iCALIBOSO 

(Male 

233  A 

X 

X 

jx 

14.5 

603050323 

(BRIAN 

MILLER 

|Male 

233  A 

1333 

X 

lx 

I4.1 

615050908 

(Donald 

(Byrne 

IMale 

233  C 

X 

X 

(x 

(4.3 

622581907 

(ERWIN 

jBAYQUEN 

IMale 

233  R 

X 

HP— i 

IX 

14.20 

623829329 

lAstor 

(Mojica 

(Male 

233  C 

X 

1 - 

|X 

(4.0 

625721795 

(JASON 

PHILLIPS 

plale 

233  A 

X 

ix 

(4.4 

031667507 

LAUREN 

TRAVERS 

Female 

234  R 

X 

X 

X 

[3.6 

082646921 

Liza 

Solon 

Female 

234  C 

X 

X 

Ix 

I3.8 

150642918 

SHALAWN 

SMITH 

(Female 

234  A 

X 

X 

i mmm 

166667854 

CELESTE 

RENWICK 

[Female 

234  A 

X 

31 

a 

EQHi 

216155550 

Kimberly 

Diver 

Female 

mm 

mam 

la 

S3 

QgflH 

224490960 

BRANDY 

"PEEBLES 

Female 

234 

H 

X 

n 

£S33 

227319491 

(Antonia 

Patterson 

Female 

234 

c 

X 

X 

X 

3.8 

229137935 

MARY 

IPPERT 

Female 

234 

A 

X 

X 

X 

4.3 

234331672 

iJULIE 

SAVILLA 

Female 

234 

A 

X 

X 

X 

4.3 

253151461 

JEJUANA 

JOHNSON 

Female 

234 

R 

X 

X 

X 

3.6 

254270747 

(AMY 

ROBINSON 

Female 

234 

R 

X 

X 

X 

3.2/3.68 

254471605 

ILATONGA 

O'NEAL 

Female 

234 

A 

X 

X 

3.2/3, 6 

254634848 

ANGELA 

JONES 

Female 

234 

R 

X 

X 

X 

4.3 

262950050 

Ijanine 

VILLOT 

Female 

234 

A 

X 

X 

X 

mmm 

278880006 

HEATHER 

RUFFNER 

Female 

234 

R 

X 

X 

X 

3.6 

282882452 

JENI 

WILEY 

Female 

234 

A 

X 

X 

X 

3.8 

322680935 

(PATRICIA 

MCCORKLE 

Female 

234 

R 

X 

X 

X 

3.4 

360747390 

STEPHENIE 

GAINES 

Female 

234 

R 

X 

X 

X 

4.5 

363820374 

LUCINDA 

MOORE 

Female 

234 

A 

X 

X 

X 

3.8 

371022442 

LAURIE 

RANDALL 

Female 

234 

A 

X 

X 

X 

3.9 

399663249 

PITA 

KLUMB 

Female 

234 

A 

X 

X 

X 

4.3 

438335318 

pari 

Taylor 

Female 

234 

C 

X 

X 

X 

4.6 

464475757 

Stefan  ie 

Walden 

Female 

234 

C 

X 

X 

X 

4.1  ~ 

474963732 

Debbi 

Scheunemann 

Female 

234 

C 

X 

X 

X 

3.5 

482029134 

LORENE 

DEVICK 

Female 

234 

R 

X 

X 

X 

3.7 

484084146 

GODIVA 

DEGUZMAN 

Female 

234 

A 

X 

X 

X 

4.4 

519086140 

Lisa 

Todd 

Female 

234 

C 

X 

X 

X 

4.6 

540338365 

ALICIA 

ESPINOSA 

Female 

234 

R 

X 

X 

X 

4.0 

542116556 

REBECCA 

MEYERS 

Female 

234 

A 

X 

X 

X 

4.2 

554554507 

MONIC 

LEE 

Female 

234 

R 

X 

X 

X 

4.2 

609721269 

VONNY 

SEROTA 

Female 

234 

A 

X 

X 

X 

4.5 

610100041 

Cynthia 

Moore 

Female 

234 

C 

X 

X 

X 

4.4 

614070751 

Brandi 

Dalton 

Female 

234 

C 

X 

X 

X 

3.5 

619249850 

Stacy  : 

Sullivan  i 

Female 

234 

C 

X 

X 

X 

4.0 

006963714  1 

CHRISTOPHE  i 

GIDDENS 

Male 

235 

R 

X 

X 

X 

4.1 

010704280  fl 

Sean 

Burke 

Male 

£ 

CO 

CM 

C 

X 

K 

X 

3.6 

Page  2 


SSAN 


052622327 

110600462 


137669309 

173661863 


FirstName 


jKazi 

IAARON 


[REGINALD 

IMICHAEL 


LastName 


[Hunter _ 

iJENKINS 


IRICHARDSON 

Ikline 


1  Gender  1 

Div# 

IGrou 

Pi 

Consent 

PreSSQ 

Post  SSQI 

111542430 

JOHN 

COYNE 

Male 

[49733936 

JOHN 

IPISTOLIS 

Male 

i 

2351C 

235IA 


35 

35 


235IC 


s 


58614057 

58635235 


164875300 

83829092 


16043486 

17785093 


21646714 


32727126 


51665179 

54685620 


55688151 


jPhillip 


Rudolph 


CHARLES 

NICHOLAS 


JUSTIN 

Gregg 


TROY 


EDWARD 


jEARL 

IMARK 


IBrady 


IDoss 


iKnight 


GUYTON 

"SINGER 


BAKER 

Manor 


jEWlsftfacE! 


49972440 


55878195 


58854617 


! 


76297826^ 

.17762376_ 

i30236755_ 

'31111250 

59590848 

59657568^ 

66596933 

66696912_ 

69592790 

71476142 

73978737 

74862977 

85296217 

91509963 


06566424 

20020982 

P97720977 


Edward _ 

JOSHUA _ 

Grant 

JUSTIN 

ALLEN _ 

Reid _ 

William 
MICHAEL 
JOSHUA 
FRANK 
WES 
"CASEY 
Anthony 
PATRICK 


ABNER 

JAY  SANTIAG 

Heather 


CLAY 

Male 

235 

R 

(BERRY 

Male 

235 

R 

TAYLOR 

Male 

235  A 

METCALF 

Male 

235  A 

Westberry 

Male 

235  C 

GEHL 

Male 

235 

A 

PERKINS 

Male 

235  A 

MARTIN 

Male 

235  R 

LOPEZ 

Male 

235  R 

HOLLEY 


CRANE 


Henderson 


David 

Doyal 

Male 

Carey 

Gameau 

Male 

JASON 

SMITH 

Male 

JASON 

DARBY 

Male 

David 

Castro 

Male 

Stoessel _ 

BEAUMONT 

Potter _ 

BOURM 

BOSSARD 

Brown _ 

Day 


A 

C 


ABRAZALDO 

ABLOG 

Derringer 


Male 

Male _ 

Female 


235 A_ 

235  R 

236  C 


Page  3 


SSAN 


98565025 


124600339 


154784818 


185646161 

04547214 


27150718 


30270302 

39397670 


57310338 


62857031 

88867569 


02806747 


FirstName 


Tania 


iDEBORAH 


TINA 


TRACEY 

(GEORGIA 


IJOLENE 


(KAREN 

IHEATHER 


(Deborah 


PATRICIA 

"JENNIFER 


(ANGELINA 


1  Gender 

|  Div  #  | 

Grou 

3 

Consent 

iPreSSQ 

Post  SSQ| 

(Davila 


IDUFF 


ISTULGA 


IlMHOFF 

ISCHOEN 


PAYNE 


COX 

"COOPER 


(Weaver 


IJONES 

Ibahr 


IHENDON 


Female 


Female 


Female 


Female 

Female 


Female 


Female 

Female 


Female 


Female 

Female 


Female 


2360 


236IA 


236IA 


236IR 

236IR 


236IA 


_236|R 

236|A 


6C 


6(R 


►20983860  LaShawna  ISwoopes 

Female 

25511051  IBEVERLY  CHADWICK 

Female 

31350965 

(Jennifer 

Zetty 

Female 

23 

6C 

32619135 

Kylie 

Cain 

Female 

23 

33730373 

AYANA 

BOURGEOIS 

Female 

23 

6A 

64414791 

LATARSHA 

KENDRICK 

Female 

236  R 

08984677 

517760145 


(Mandy 

BRANDY 


James 

BEAGLE 


pernale 

Female 


120988077 

Kelly 

Troutt 

Female 

236  C 

25696126 

MELISSA 

BARELA 

Female 

236  R 

29536789 

VICKIE 

KARTCHNER 

Female 

236  R 

33822382 

Tara 

Hayes 

Female 

236  C 

35060532 

April 

McDonald 

Female 

236  C 

2026971 

SUSAN 

BERGSTROM 

Female 

236 

R 

2250929 

SHEILA 

WINONA 

Female 

236 A 

3024941 

ELIZABETH 

STEWART 

Female 

236  R 

1 

i 

i 

1 

I 

\m 


44219405 


58992533 


91668346 
95907581 
1 3604 1 79 
16961 147_ 
21348806^ 
01725988 
36460150 
037527379_ 
097745538 
180608755 
227379153 
232257468 


48490686 
48491184 
50690 120_ 
55636863 
56063246 


VTiTT 


IAMANDA 


JRobyn 


MELISSA _ 

ESTIVALIZ 

TARADEEN 

TAMMANY_ 

"CHRISTEN 

"BRIAN 

Mario 

Brett _ 

SEAN _ _ 

JOSEPH 

Stephen 

WILLIAM 


Rayfield 
ANTONIO 
KENNETH 
HECTOR 
Stacey 


KAMPMILLER  Female 


Nelson 


LEWIS 

QUINTANA_ 

"MORGAN 

ATIENZA 

BRANCH 

"TOMASELLI 

Belluscio 

Albee _ 

WILLIAMS 

MCKELVEY 

Kidd _ 

CALVERT 


Gordon 
ROBINSON 
GREEN 
RODRIGUEZ 
Mobley 


Female 


Female 

Female 

Female^ 

Female 

Female 

Male 

Male 

Male 

Male 

Male 

Male 

Male 


M 

a 

M 

a 

M 

al 

M 

al 

M 

al 

237  C 

237 A 

237 A 

Administration  Records 


8/27/98 


SSAN 

I  FirstName 

!  LastName 

|  Gender  |  Div#  |  Group  |  Consent 

1  Pre  SSQ I  Post  SSQ|  Exam  j 

257610080 

IJASON 

IMOSS 

(Male 

i  -  237IA 

lx 

IX 

IX 

14.6 

260639093 

IBRIAN 

IBELCHER 

IMale 

1  237iR 

lx 

ix 

IX 

14.3 

335621921 

IDEMOND 

(WIGFALL 

(Male 

237IA 

ix 

lx 

IX 

13.8 

352766571 

COLIN 

(SPRINGER 

Male  • 

j  237IA 

X 

1 

lx 

IX 

|4.5 

353762840 

JAIME 

OCHOA 

Male 

237 

1A _ 

X 

X 

ix 

[4.8 

411412271 

Imichael 

ITURPIN 

Male 

237IA 

X 

X 

lx 

W.7 

435531753 

BRANDON 

LORENTZ 

Male 

237 

'R 

X 

X 

ix 

14.2 

438557878 

CHARLES 

JOHNSON 

Male 

237 

’A 

X 

X 

-j - 

IX 

4.1 

442743716 

MICHAEL 

ISTEWART 

Male 

237 

1R 

X 

X 

Tx 

ki 

443940347 

Tjanos 

IsAGI 

Male 

237 

1R 

X 

X 

ix 

1^5 

452695478 

jBRUCE 

HIMELRIGHT 

Male 

237 

|a 

X 

X 

Ix” 

4.2 

457613198 

JASON 

KOSTER 

Male 

237 

lR 

X 

X 

X 

|4.1 

466777265 

Ikhammay 

INORKEO 

Male 

237 

IR 

X 

X 

X 

3.4 

467416882 

|DANIEL 

REYNA 

Male 

237 

(A 

X 

X 

X 

4.5 

474155312 

|JOSEPH 

ROSS 

Male 

237 

|R 

X 

X 

X 

4.0 

476029798 

(Adam 

Martinson 

Male 

237 

!c 

X 

X 

X 

4.6 

506139016 

Anthony 

Staman 

Male 

237 

IR 

X 

X 

X 

4.6 

520901080 

Christopher 

Gibson 

Male 

237 

C 

X 

X 

X 

4.3 

524470521 

JOHN 

THATCHER 

Male 

237 

A 

X 

X 

X 

4.6 

526810212 

Christopher 

Matthews 

Male 

237 

c 

X 

X 

X 

3.6 

530936406 

ARNOLD 

MANCIA 

Male 

237 

R 

X 

X 

X 

4.6 

538967932 

JonPaul 

Marquis 

Male 

237 

C 

X 

X 

X 

4.6 

540253293 

BRIAN 

GRUBBS 

Male 

237 

R 

X 

X 

X 

4.4 

563537061 

DAMON 

JENKINS 

Male 

237 

A 

X 

X 

X 

4.4 

566799763 

MICHAEL 

ASHWORTH 

Male 

237 

R 

X 

X 

X 

4.6 

604387047 

L 

Gueorguieua 

Male 

h* 

CO 

CM 

C 

X 

X 

X 

4.6 

606054477 

[Andrew 

Crawford 

Male 

237 

C 

X 

X 

X 

4.8 

606980444 

Joephaz 

Sanchez 

Male 

237 

c 

X 

X 

X 

4.2 

607014689 

(DOMINIC 

VEAL 

Male 

237 

A 

X 

x 

X 

3.2 

613741803 

ICELERINO 

MERCADO 

Male 

237 

R 

X 

X 

X 

3.2 

027585631 

JESSE 

ISAYER 

Male 

238 

R 

X 

X 

X 

4.3 

082603709 

Jeffrey 

DelGrange 

Male 

238 

C 

X 

X 

X 

4.6 

082869967 

SHLIEMANN 

GERLUS 

Male 

238 

R 

X 

X 

X 

3.9 

092724434 

RICHARD 

RABAGLIA 

Male 

238 

A 

X 

X 

X 

3.9 

102845463 

HATIM 

TALIBI 

Male 

238 

a" 

X 

X 

X 

4.0 

116862352 

CHARLES 

ADUFOKUO 

Male 

238 

A 

X 

X 

ix 

4.8 

117600388 

KARAM 

CUMMINGS 

Male 

238 

A 

X 

X 

X 

4.7 

124724689 

PABLO 

FINELLI 

Male 

238 

A 

X 

X 

X 

4.6 

128649038 

TERRENCE 

SINGLETON 

Male 

238 

R 

X 

X 

X 

3.7 

205483391 

DURON 

DEANSMITH 

Male 

238 

R 

X 

X 

X 

3.9 

231415921 

Rodney 

Boyd 

Male 

238 

C 

X 

X 

X 

4.2 

234195524 

David  i 

Oliveto 

Male 

238 

C 

X 

X 

Ix 

\4.2 

243572953 

ROBERT 

MORRIS 

Male 

238 

A 

X 

X 

X 

3.8 

244153504 

COREY 

NESBIT 

Male 

238 

R 

X 

X 

X 

4.2 

252572948  ! 

SHEABIAN 

MAPP 

Male 

238 

R 

X 

X 

X 

4.0 

1263855201 

Paul 

Brooks 

Male 

238 

C  ; 

X 

X 

x 

4.2 

,362888195 

William 

LEMIEUX 

Male 

238 

R  f 

K 

K 

X 

3.9 
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SSAN 


74885896 

01176347 


38553157 


44887828 


49775825 

54834818 


54937900 

95867580 


12948854 


28539383 


15 

81 


42963225 


8530082 


51474418 


51692710 


54691845 


26 

64 


79827691 


89786914 


02887400 


10026164 


11053196 


14808064 


15188707 

15427249 

17568991 

24056227_ 

24981125 

35071198 


FirstName 


IROBERT 

IROBERT 


LastName 


IHOLLINS 

ISEARCY 


Gender  Dlv#  I  Group!  Consent  Pre  SSQ  Post  SSQ 


iMale  i 


REY  NAZARE 


Joseph 


IBRENT 


ROSCOE 


ELROY 


David 


Jenner 


JIE 


NICKY 


CHRISTOPHE 


ARESTIDES 


GABRIEL 


JOHN 


Do  Soo 


MORONI 

ADOLFO 

JORGE _ 

Joshua _ 

Marlon 

Dien 


Weatherford 

DeGuzman 

Hua 


CD  |  CO 


Pre-stress 


8/27/98 


SSN 


01725988 


06963714 


10704280 


27585631 


29641195 


31667507 


36460150 


37527379 


42703652 

52622327 


65728767 


62869967 

|RM 

238  4 

1 

188662527 

C 

|M 

233  2 

2 

S2724434 

AO 

|M 

238  2 

2 

C 

iF  .... 

236  1 

2 

97745538 


102845463 

110600462 


116862352 

117600388 


124600339 


124724689 


128649038 


137669309 


137865156 


150642918 


16155550 

24490960 


27150718 

27272077 

27319491 

27379153 

27418850 


154784818 

AO 

F 

166667854 

AO 

F 

173661863 

RM 

M 

180608755 

RM 

M 

185646161 

RM 

F 

186680564 

C 

M 

204547214 

RM 

F 

205483391 

RM 

M 

211542430 

AO 

M 

C _ F 

AO  F 


AO 

F 

RM 

M 

Pre-stress 


8/27/98 


Page  2 


SSN 


30270302 


30410090 


31415921 


34195524 


34331672 

35355563 

39397670 


45041869 

48490686 


48491184 


49733936 


50690120 


57610080 


58614057 


58635235 


60639093 


62857031 


62950050 

63855201_ 

64875300 

78880006 

82882452 

83829092_ 

88867569 

91829141 

02806747 

13967398 

16043486 

16989493 

17785093 


Gender 


IF  1236 


Q1 


13  14 


132257468  |AO 

iM  1 

133316233  |RM 

|M  i 

3  13  |2 


140556411  |C 

M 

143572953  AO 

M  | 

44153504  RM 

M  | 

1  1 


53151461 

RM 

F 

234 

1 

54270747 

RM 

F 

234 

3 

54471605 

AO 

F 

234 

4 

54634848 

RM 

f" 

CO 

_CVJ 

2 

55636863 

RM 

M 

1237 

1 

Q14 


15  15 


1 

5 

1 

5 

15 

|5 

Pre-stress 


8/27/98 


Pre-stress 


8/27/98 


SSN 


30681212 


30724123 


32727126 


Gender 


IRM  iM 


IRM  |M 


60669734 

60747390 


61745659 

62888195 


63065998 

63820374 


63925421 

71022442 

74885896 


01176347 


11412271 


,  IM 


1AO  |M 
'RM  |F 


C _ |M_ 

|RM  M 


lAO  M 
|AO  F 


C _ [M_ 

AO  |F 
RM  IM 


lAO  IM 


23131466 

25511051 


31350965 

32619135 

33730373 

33733185 

34596439 

34656760 

34698831 

35531753 

37211754 

37357405 

38335318 

38435862 

38553157 

38557878 


0929027 
2743716 
1443940347 


M 


AO  M 
RM  F 


C _ F 

C _ F 

AO  F 


R  M  _[M 

rm"  m 

RM  M 
AO  M 
RM  IM 
AO_M 
RM  M 

C _ F_ 

AO  M 

C _ M 

AO  M 


RM  M 
RM  M 
RM  M 


1235  13 


51665179  |AO 

M 

in 

CO 

c\j 

4 

52766571  |AO  i 

M 

237 

4 

5 


5 


13 

li  li 


1  |1  io  |2 


13  13  12 


IQ 


1235  |4 


1235  3 
|234  2 


233  |5 
1234  |l 
238  12 


|233 

2 

CM 

14314654 

C 

M 

18156086 

RM 

M 

20233024 

RM 

M 

238 


237  3 


238  4 


1238  2 


235  3 

236  1 


1 

3 

2 

3 

3 

3 

1 

1 

2 

3 

\mv 

mg 


235  3 
237  4 
237  2 


4  _ 5_ 

5  _ 1_ 

5  5 


3 _ 

2 


4 _ 5^ 

3 _ 2_ 

2  2 


_I4 _ 13_ 

H  II 


3 

12 

12 

|3 

10  14 


14 


Q14 


J5 _ 4 

II  !1 


15  |1 


15  |4 


E3| 


1  _ 1 

2  4 
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Pre-stress 


8/27/98 


SSN 

K25? 

i  Gender 

44887828 

|RM 

IM  1 

45840185 

IRM 

|M  | 

\rrm 


\r 


9775825 


49972440 


ic  I 


ic  I 


IRM  |M 


1238  2 


13  |3- 


13  12 


i3  |3 


|3  13 


Q14 


14  13 


EEll 


54834818 


55878195 


57613198 


57657206 


58854617 


63796866 

64414791 

64475757 


82029134 


83903771 


84084146 

87946653 


20988077 


24470521 


C 


RM  M 


lAO  M 


|AO  |M 


|RM  M 
RM  F 
C  F 


M 


RM  F 


IAO  M 


AO  F 
IC  M 


195867580 

RM 

M 

.06139016 

RM 

M 

.08984677 

C 

F 

12948854 

AO 

M 

15766180 

RM 

M 

17119820 

RM 

M 

17760145 

AO 

F 

17762376 

AO 

M 

17863523 

RM 

M 

19086140 

C 

F 

20889977 

RM 

M 

20901080 

C 

M 

AO  M 


238  3 
238  b 


1237  2 


233  5 


1235  3 


236  3 
234  12 


235  2 


234  |2 


233  1 


234  2 


ieeeie 


1  1 


3 


164914268 

AO 

M 

235  1 

166777265 

RM 

M 

237  4 

167416882 

AO 

M 

167516569 

C 

M 

174155312 

RM 

M 

237  4 

174963732 

C 

F 

234  1 

M 

237  4 

1  1 


3  3 


|1 

1 

3 

3 

14  |4 


14 


4 _ 4 

3  3 


4 _ 4 

3  3 


5 


Pre-stress 


8/27/98 


Page  1 1 


Pre-stress 


8/27/98 


iFEfl 

IEE3 

\um 

\nm 

lEna 

3 

ii 

ii 

13 

14 

1 

ii 

ii 

(5 

15 

ii 

15 

h 

12 

ii 

1 

|1 

3 

14 

s 

S 

s 

3 

io 

Is 

S 

1 

3 

5 

4 

2 

3 

5 

4 

5 

2 

1 

4 

4 

4 

3 

3 

3 

4 

2 

4 

2 

IS 

3 

0 

3 

0 

3 

1 

5 

2 

2 

4 

3 

4 

3 

3 

S 

m 

4 

4 

Is 

s 

K _ 

5 

15 

i 

i 

S 

IP _ 

2 

3 

3 

3 

IS 

IS 

2 

2 

5 

5 

3 

1 

1 

3 

3 

91 

3 

3 

4 

3 

s 

1 

1 

3 

3 

s 

1 

3 

5 

3 

2 

3 

s 

2 

0 

4 

4 

4 

s 

s 

2 

3 

1 

Is 

s 

1 

5 

5 

5 

2 

2 

5 

5 

4 

2 

1 

4 

4 

3 

1 

1 

5 

4 

4 

1 

1 

1 

1 

0 

0 

0 

0 

IjH 

1 

2 

3 

4 

1 

1 

4 

3 

51 

2 

1 

3 

2 

K _ 1 

2 

1 

3 

3 

3 

3 

3 

3 

flnl 

2 

4 

4 

5M 

Sin 

Sts? 

9H 

^Bi§ 

9n 

on 

51 

m 

rn 

ini 

3H 

s 

35 

s 

an 

si 

s 

s 

s 

nil 

p 

o  1 

s 

II 

Page  12 


SSN 


27933369 


28539383 


29536789 


55 


30936406 


I2Z5II 

Gender 

m 

Q1 

3  |3 

13 

13  15  11  13  | 

3 

CO 

13  |3 

31111250 


33822382 


F^il?Tli^lI^lE3E3El|g5|[ElElEIllEniEigRFlFmFTHI 


1238  !2  ! 


1236  11  14 


1233  |4 


235  |4 
1236  |3 


35060532  |c  F 

236 

3 

38312181  IRM  M 

238 

3 

38967932  |C  M 

237 

3 

40253293 


40338365 


42026971 

42116556 


2250929 


42963225 


IRM  M 


|RM  F 


M 


IRM  IF 


1233  3 


1238  2 


g 


3176102  RM 

M 

1233 

4219405 


45578423 

48530082 


m 


|233  12 


236  2 


233  3 


238  3 


51474418 

AO  | 

M 

51692710 

RM 

M 

53794902 

RM 

M 

2 


5 


5  3 


3  5 


5 


0  2 


5  2 


3  3 


3 


3 


1  1 


2 


3 

3 

1 

3 

3 

2 

1 

1 

1 

!3 

b 

Pre-stress 


8/27/98 


Page  14 


SSN 

Gender 

!QS9E!lE3El]EI]ESiEIlEEaEIlEI]EII3En]EIQElB]l 

Q14 

85296217 


189786914 

IAO 

IM 

238  13 

191509963 

iRM 

IM 

235  2 

191668346 

AO 

236  1 

195907581 

AO 

F 

236  2 

101645120 

RM 

M 

233  1 

02886386 

AO 

M 

233  1 

|4  |0 


13  13 


|1  |0  II  |3 


3  12-  5 


02887400 


03050323 


06054477 


06566424 


06980444 


07014689 

AO  M 

09721269 

AO  F 

10026164 


11053196 


13604179 

RM  F 

13741803 

RM  M 

15050908 


15188707 


6 

15427249 

AO  M 

238  1 

2 

3 

3 

4 

m 

2 

2 

■■■■ 

4“1 

irn 

6 

16961147 

RM  F 

236 

4 

2 

3 

3 

5 

5 

3 

3 

2 

3  J4 

17568991 


Pre-stress 
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Page  17 


'age  18 


SSN 


01725988 


B'lfcMsiKSE 


27585631 


29641195 

31667507 


Gender 


IRM  IM 


|R 


1C  | 


IRM  IM 


lAO  IM 
|RM  F 


1237  14 


1233  12 


(238  12 


233  14 

234  k 


14 


5 


15 


2 


15 

IT- 


10  12 


13 

5 

14 

5 

12 

5 

14 

5 

5 

15 

15 

5 

|5 

5 

42703652 


52622327 

65728767 


82603709 


82646921 

82869967 


88662527 

92724434 


97745538 


98565025 


102845463 


IRM  |M 


C _ |M_ 

"AO  |M 


1C  IM 


JC _ F_ 

"rm  Im 


C  |M 

"ao  m 


IRM  |M 


AO  |M 


3 


235  14 
"233  U 


1238  |4 


234  4 
"238  4 


233  3 
"238  |4 


2_ 
|238  |4 


13 _ h_ 

±  Ig- 


5 


5  2 


5 _ 5_ 

"3  S’ 


117600388 

AO 

M 

238  5 

124600339 

AO 

F 

236  0 

124724689 

AO 

M 

238  5 

128649038 

RM 

M 

mi 

137669309 

AO 

M 

235  2  , 

137865156 

AO 

M 

233  4 

150642918 


154784818 
166667854_ 
1 73661 863_ 
180608755 
185646161 
186680564 
204547214” 
205483391 
211542430 
16155550 


AO  F 
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